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Preservice Teachers' Development of Pedagogcal UnderstancSngs and Epistemological Frameworks 

Chris L. Lav^rence 

Abstract 

This study presents one method for assessing preservice teachers' development of broad 
pedagogical understandings in relation to an informal reasoning framework and instmctiooal 
processes. The participants in this study were 72 undergraduate preservice teachers who 
completed a set of 1 1 open-ended planning tasks, self/course evaluations, and a measure of 
learning orientation for a course in educational psychology. The tasks faditated the applrcation of 
both broad and specific pedagogical concepts, previous knov^edge and beliefs about classrooms, 
subject matter knowledge, and knowledge about classroom planning. The way participants 
conceived, developed, and structured the tasks is described in two sets of holistic scoring 
rationales, representing acro.^ tasks and within tasks analyses. High intenrater reliability (r - 
.89 - .91) and internal consistency (a » .85) was obtained when applying the Levels of 
Pedagogical Understanding (task specific rationales) to participants' responses. An analysis of 
individual profiles resulted in an initial formulation of a typology for epistemological frameworks. 

This study, in its broadest sense, describes how preservice teachers in an educational psychology 
dass make meaning of pedagogy. Goodad (1 990) thinks there is an overwhelming dominance of dassroom 
teaching for lovvHevel intellectual skills (he sees that even the basics are not being successfully taught in 
dassrooms) and that the problem can be traced back to teacher preparatkxi progranrB. Studies have 
indicated that teachers are not adequately prepared to ©igage in complex reasoning and didt higher-level 
thinking and discourse in classrooms. More complex understandings of cognitive, sodal, and moral aspects 
of children's growth or how to adequately fadlitate and plan for children's growth are often not explored 
and not taken up as a primary purpose in research. This nrwiy be a result of little emphasis in studes on how 
teachers devdop understandngs of pedagogy and about how teacher preparation programs can aicourage 
development through instructkxi. The domain of educational psydxilogy has become enomxDUs, the research 
prolific, and the purposes varied. Students often see educational psychology, and many other required 
education courses, as hoops they had to jump through in order to g-aduate, become certified, and start 
teadiing. A large part of the problem is that students do not fee! they are learning anything useful or 
rdevant to actual teaching practice or not experiendng forms of personal growth. Students see their 
coursework as relevant when the instmctional content and process are nrjeaningfully connected to their 
development as teadiers. One of the nrxjst dfficutt tasks fa teacher educators is to help indviduals 
broaden their perspectives and develop deeper pedagogcal understandngs. How does one teach courses like 
educatfonal psydiology with complex and dverse content and focus on higher-order thinking? To answer 
this questton the following questtons must also be addressed How does the instmctional process interact 
with students' ability to make deeper nr>eanings of content?. How can dfferent levels of understandng be 
taken into account to promote development?, and What theoretkal frameworks wouW support these 
concerns? 
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I have found that each of my students has some epistemolosM and pedagogpcal understandings 
v^ich guide them whether they are simple or complex, limited or broad, disparate or Integrated, and many 
manifestations In between, h respond«ig to open-ended tasks developed for this study, some individuals 
give typical behavioral solutions even when directed toward cognitive solutions, often Incicating they would 
drect their own students' conceptions by focusing on right/wrong answers or acceptable/unacceptable 
classroom behaviors. Other Individuals develop plans indicating a dynamic and fairiy welkteveloped 
conception of thinking and of instruction's rote in pronrwting and dwsrosing students' development As with 
Ahlquist and O'Loughlin (1 990), this study reveals that some Individuals have an overly authoritarian 
stance toward teacher's and students' classroom rotes. When asked to plan for preventing dsclpline 
problems, these individuals give answers that reflect a tight control on what can occur in the classroom 
even to the point of limiting the learning and social interactions vs*iich could take place. These indivickjals 
often give answers representing reactive approaches to dscipline, what they will do after a discipline 
problem occure and provide no proactive approaches. Other individuals promote proactive measures such as 
engaging students in meaningful teaming, being sensitive to students' emotional needs, focusing on more 
natural consequences, and structuring the classroom so tools for teaming are available to help lessen 
discipline problems and spend more classroom time on meaningful activities. They will often describe an 
entire system, first setting up a positive environment, getting their students involved in nwiking decisions 
about rules and consequence for the unacceptable classroom behaviors, and i^borating upon their own rote 
in the classroom. Of course, many individuals portray different mixtures of conceptual understandings and 
fall somewhere between the extremes described here. A goal of this study was to more cleariy delineate a 
continuum of pedagogical understandings and epistemological frameworks of the preservice teachers who 
participated 

This study will present a means of assessing preservk:e teachers' development of pedagogcal 
understandings and epistennological frameworks through semi-structured, ongoing, and authentic tasks 
within the time frame of one course (in the context of educational psychology). The tasks provide for 
individual variation in the construction of meaning, accounting for not only individual variations in style but 
also In age and subject matter concerns. The method of assessnrient focuses on generaSzabte themes and 
developmental continuums related to broad pedagogical understandings. A general th^oretteal framework 
which delineates a nxxtel of informal reasoning, assessment of tevels of understandings based on quality, 
and instructkxial processes whkh encourage the devek)pment of preservice teachers' f-«5dagogical 
understandings will be presented. The framewori( is viewed in light of the nr>ore global considerations of 
teacher socialization processes and teachers' personal growth. An individual's personal growth and learning 
approach are viewed as integral to their devek^pment of pedagogical understandings. 
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A Framework for Informal Reasoning and Assessing Levels of Pedagogcal Understandings 
Pintrich (1 990) reviewed the literature on psychological research and discussed the implications of 
this research for teacher education. He identified these four specific areas of concern as important to 
future research on teacher education: teachers* (a) knowledge, (b) thinking and problem solving, (c) 
metacognition and self-regulatton, and (d) motivation. Each of these concerns however, couW be dePned in 
many different ways, i.e., there are many definitions of thinking and problenr«olving currently in use. In 
the following discussions, the concerns addressed are preservk:e teachers* (a) pedagogical understand ngs, 
(b) Informal reasoning, (c) reflectivity, and (d) learning approach. The paradgm for teacher learning is 
constmctivist, based on conceptions of how reasoning deveksps and how individuals make meaning of 
pedagogy. 

A key factor in many studies on teacNng cogiition lies in their emphases on how cognrtk)n 
develops or what knowledge develops. A number of researchers have delineated the importance of 
teachers* conceptual structures, understandings, and systems and the limitations whk:h occur when 
these are too simplistic or under-developed or how they are facilitated when complex and well- 
developed. The quality of teachers* conceptual systems can help them account for the inherent 
complexities in classrooms, plan for flexible activity, and find deeper meaning when interpreting 
dassroom events (CaWerhead, 1 981 ). Livingston and Borko (1 989) propose that successful 
improvisational teaching is facilitated through an extensive network of interconnected and accessible 
schemata. They find that the quality of a teachers* knowledge is a major influence in improvisational 
teaching. The schema or cognitive structures, related to knowledge of teaching, are important to 
consider when planning instruction for preservice teachers to help individuals assimilate, retain, and 
process information related to teaching (Livingston & Borko, 1 989; Peterson & Comeaux, 1 987). 
Knowledge and Icnowledge structures w\\ affect the way a teacher vwil account for the complexities of 
the classroom and interact with their reasoning about complex events. Most studies on teacher 
cognition also advocate instruction in teacher education based on an individuals* level of development 
(Livingston and Borko, 1989; Peterson & Comeaux, 1987; Berliner, 1989). 

Prevk)us studies have attempted to define what knowledge is import by studying 
experts/experienced teachers, but these are lacking in pedagogical content structures, devetopmental 
continuums, and instmctional theories which can be directly applied to preservice education. Missing 
from current studies on teacher cognitk)n (with a few exceptions, e.g., Anvnon & Hutcheson, 1 989) is 
a description of what knowledge is important to know and how knowledge structures are organized To 
assess progression in teachers* developnf>ent, the noethod must account for dfferent levels of 
understanding or levels of competence, guided by conceptions of higher-order thinking (Frederiksen, 
1984; Wiggins, 1989). 
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The method in tNs study, was to develop a content stnjcture based on broad pedagogical thenr>es 
which are persistent In (or ennerged from) the participants' responses, themes such as teacher's and 
students' rdes in the classroom and the content and conceptual stmcture of dassroom plans. Within 
each theme a-e levels of organization of know(edge or how this knov^*edge Is structured. Pines (1 985) 
explains the tenm cognitive structure, vAich is concomitant vwth how pedagogical knowledge is 
structured, in the following. 

What is the meaning of coonitive structure ? The words give us important dues. Coonitive 
means "of the nind, having the power to know, recognize, and conceive, conceming personaly 
acquired knowledge, " so cognitive structure concems indvidual's ideas, meanings, concepts, 
cognitions, and so on. Stmcture refers to the form, the arrangement of elements or parts of 
anything, the manner or organization; the emphasis here is not on the elements, althou^ they 
are inportant to a stmcture, but on the way those elements are bound together ... What binds 
them together are "relations". So too does their structure. The meaning that an individual gives 
to a particular word, and the complex conceptual frameworic that an individual possesses which 
makes him or her knowledgeable in a particular area boch depend significantly on relations, (p. 
101) 

Pedagogical understandngs become more complex in both form and functkxi (relations) as developn^t 
occurs. Pedagogical reasoning is facilitated through reference to broad pedagogrcal understandngs and their 
interrelationships. Well-developed pedagogical understandngs and eptstenrxJogical frameworks, would not 
only akJ in preservice teachers' interpretation of infonnation on and devetopment of knowledge about 
teaching and learning but could also help guide their actual dassroom practice. 

Peterson & Comeaux (1 987) have drawn upon information processing and expertise literature in 
conceptualizing the function of schemas and general or fundamental principles as organizing agents within a 
domain of knowledge. In previous work on levels of cognitive complexity (McDaniel & Lawrence, 1 990), we 
found that indviduals who find deeper meanings in an issue or situatkxi, organize their ideas around major 
themes and broad conceptions (in this study, pedagogfcal themes and context spedfic conceptions). These 
indviduab are able to see the complexity in a situation and organize the many facets in a manner which 
simultaneously maintains the conrplexity and also simplifies the reasoning process by fomiulating priorities 
around what is valued, ethical considerations, and/or the most subsuming ideas/concepts. The organizatkxi 
or stmcture of the dscourse facilitates (or limits) the Integration of ideas with previous knowledge, new 
information, and larger bodes of knowledge (worid knov\4edge, knowledge of systems) ♦ 

McDanid and Lawrence's (1 990) rationale describes five levels of complexity which were apparent 
In indvidual's responses to open-ended and complex situations. As the complexity of thinking increases, the 
causal connections also increase as does the quality of the relationships fonned. Validty for these 
stmctural properties is supported through a dscourse analysis conducted by Lawrence and Stewart 
(1 990). The scoring rationale is presented next. 
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Levels of Cognitive Complexitv . 



Level 1 : Unilateral Descriptions 

Simplifies the situation. Focuses on one idea or argument. Does not identify altemattves. Brings in no 
new infomrwtion, meaning, or perspectives. Makes "good^)ad*' and "either-a'*' assertions. Appeals to 
authority or sinfY>le rules. Information Is simply paraphrased, restated or repeated. 

Level 2: Simplistic Alternatives 

Identifies sinple and obvious conflicts, but the conflicts are not pursued or analyzed. Develops a 
position by dismissing or ignoring one aitemative and supporting the other with assertions and simple 
explanations rather than through deeper assessment of the situation. 

Level 3: Emergent Complexity 

Identifies more than one possible explanation or perspective. Complexity Is established and preserved. 
New elements are introduced Supports position through comparisons and simple causal statements. 

Level 4: Broad Interpretations 

Uses broad ideas to help define and interpret the situation. Manipulates ideas within the perspective 
established. Has a clearly recognizable explanatory theme. Ideas are integrated into "sub-assemblies" 
each supporting a component of the explanation. 

Level 5: integrated Analysis 

Restructures or reconceptualizes the situation and approaches the problem from a new point of view. 
Constructs a network of cause-and-effect relationships. kJeas are integrated and extrapolated. Anives 
at new interpretations by analogy, application of principles, generalizations, and worid knowledge. 
Constructs organizing framewak, sketches connections, and predicts consequences (McDaniei & 
Lawrence, 1990, p. 78). 

This view is consistent with an informal reasoning framework. The theoretical background McDaniei 
ar>d Lawrence (1 990) discuss integrates other influences on the developnient of higher-order thinking, such 
as Perry's nrKxiel of intellectual and ethical developnrient (Perry, 1970) and an individual's reflective 
attitude (Siegd, 1 989). This background is relevant to conceptualizing preservice teachers' development of 
pedagogical understandings and epistemological framewaks ^thin instruction. The following sections 
elaborate this background in terms of their relevance for teacher educatkxi and promoting growth in 
preservice teachers. These sections are Informal Reasoning Frameworks, Adult Developmental Continuums: 
Building l ^'-rjoal Frameworks for Evaluating Knowledge, Teacher Reflectivity and Individual Learning 
Processes, Broad Considerations for Instnictional Processes, and Developing Open*Ended Tasks. 

Informal Reasoning Frameworks 
Informal reasoning problenr>s or situations have the following characteristics (from Galotti. 1 989): 

- Some premises are implicit, and some are not supplied at all. 

- Problems are not self-contained. 

- There are tyoically several i jssible answers that vary in quality. 

- There rarely exist established procedures for solving the problem. 

- It is often unclear whether the current best solution is good enough. 

- The content of the problem typically has potential personal relevance. 

- Problems are often solved as a means of achieving other goals (p. 335) 
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As would be expected, a description of the stmcture of formal reasoning problems sounds much like 
the opposite end of the continuum: premises are given, problems are $elf-<xxitained and bounded, there are 
correct answers to the problem, and there are established rules for solving the problem. The framework 
for r masoning which is advocated serves as the basis for how the constmct of reasoning (critical blinking 
or higher-order thinking) is defined and measured. Fomnal reasoning has typically implied solving problems 
that are well-defined and have a coo'ect answer whereas everyday/informal reasoning has implied solving 
problems that are ill-defined and have many possible responses. 

Situations w4iich teachers must face when employing reasoning or planning for instruction are more 
similar to infomial reasoning situations than formal reasoning situations (where there are correct 
answers, rules for solving the problem(s), and problems are bounded). This is not to say that all problenns 
are unbounded in teaching, but rather that some more strictly bounded problems occur within niostly 
unbounded classroom situations. While teachers work within different premises (and under certain set 
conditions or guidelines, such as mandates for curriculum, etc.), the day-to-day activity is primarily filled 
with many decisions and inputs, all of ^k:h have personal relevance to both the teacher and the students. 
Teachers make sets of decisions regarding students, events, or instructional activitites, etc. which do not 
often coincide with an established procedure because there are not set procedures for handling al mixtures 
of possible dassroom variables and occurence. In adcition, independent problems are not solved or plans 
are not made as an end point in teaming or as a goal in themselves, but as part of larger educational goals 
and concerns. For instance, curriculum decisions are made on many different levels, nrx)ro specific 
decisions are in some way connected to n>ore global and long term concerns. 

Assessment of informal reasoning is developed In accord with the definition of constructs, the 
irrplied structure of the problem to be solved, the nature of the task to be performed, and the resultant 
individual meanings which are constructed. The model for infomial reasoning already described was utilized 
in conjunction with pedagogy and subject matter content to fomruiate tasks and assessn^ents v\*iich can be 
integrated into instruction. Tasks provided some structure and guidance (similar to premises) but did not 
define ridgid problems to be solved. In other words, the tasks were not self-contained, but were open to 
individual responses varying in quafity and could be connected to broader concerns such as an individual's 
instmctional goals and subject matter content. Assessment can foe js on the range of individual and group 
understandngs elfcited through the task. Further discussion of assessment of individual responses and task 
conskierations will be presented in a subsequent sections. Immediately following rs a discussion of adult 
developnwntal continuums in relation to epistemological frameworks and levels of understanding. 

Adult Developmental Continuums: Building Personal Frameworks for Evaluating Knowledge 
Peny*s (1 971 ) theory or scheme of intellectual and ethical development in adulthood, describes 
nine qualitatively different positions (similar to developmental stages) of individuals. In this constructivist 
model; the focus is on indh/iduals building their own frameworics for evaluating knowledge and how they 
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integrate these eplstenx>logical frameworks with their own personal and n\orz\ identities. In lower levels of 
cognitive complexity and adult intellectual and ethical development, reasons or support are often missing 
from the blanket assertions an individual will make or the ideas they will promote. There is oin appeal to 
authority and set/simplistic answers. In the higher levels of these two schemes, individuals base decisions 
or judgements on what is viewed as important or what is valued. Judgements and decisions are supported 
vA\h explanations and evkience. In the same sense, lower levels of pedagogical understanding represent 
simplistic ideas about children's learning, about classroom activities, and about the classroom environment 
yMdh are asserted and pron-ioted, but which are not supported through rationales aid which do not take 
many classroom or learner variables Into consideration. There is also an appeal to set and simplistic 
answers for teaching, leaming, and the set-up of the classroom environment. Higher levels of pedagogical 
understanding represent ideas which take the complexities of the classroom and of learners into account. 
Solutions are arrived at or judgements are made which take multiple considerations into account, are 
supported through rationales, and are elaborated upon in temis of instructional and developmental 
considerations. 0*Loughlin (1 990) sees a connectk)n between a teacher's intellectuai and ethk:al g-owth and 
how they will view their role and their students' roles in the classroom. Some of the tasks in this study 
intersect with these same concerns, some more heavily loaded on teadier and student roles than others, in 
general, lower levels of pedagogical understanding portray the teacher as authoritarian and the student role 
as passive whereas in higher levels the teachers* role is more facilitative and the students' role is active. 
These roles often contribute to a vision of the classroom environment and the types of instruction and 
social interactions which will take place. 

Pintrich (1 990) points out that "... models for teacher development will have to allow for 
individual, contextual, and developmental variation but not be so dispersive theoretically as to be useless 
for generalization or as guides to practice" (p. 850). Perry's nrxxlel is not directly applicable within the 
time frame of a course because it describes gross developnr)ental changes. It is wore directly applicable 
over the time frame of an undergaduate teacher education program. McDaniel and Lawrence's (1 990) 
nnodel of cognitive complexity, while applicable within the time frame of a course is essentially content 
free and is only applicable in certain contexts. However, the way understandings and frameworks develop 
in these schemes, alor j with conceptions of quality and form, is useful in conceptualizing, encouraging, and 
assessing teacher developnr>ent. An informal reasoning framework in combination with an emphasis on 
measuring levels of understanding (like cognitive complexity where the quality of understandings can be 
ascertained) can contribute to these qualities in delineating one model of teacher devek)pment. This type of 
nrxxlel allows for individual variation in constructions and the evaluation of generalizable qualities. Similar 
rationales have been constructed for this study which assess Levels of Pedagogical Understanding. 

There are sinr^larities between Peny's conception about the building of epistemoiogical frameworks 
and the way preservice teachers buikl epistenfX)logical franr)eworks based on pedagogy. For this study, I 
have used the following conception of epistemoiogical frameworks. Epistemolo^cal frameworks are the 
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complex sum of one's pedagogical understandings, conceptions, or belefs. in other words, epistenrK3iogies 
are not Just the additive sum of understandings, but are a web of understandings which are inextricably 
connected in some fashion. For some, pedagogical undetstandngs play a symbiotic and synergistic role in 
fomuilating dynamic epistenwlogical frameworlcs. For others pedagogical understandings can be somewhat 
discrete from one another and can limit each other. The epistemological frameworks are therefae 
simplistic or are disparate* Beliefs, conceptions, and pedagogical understandings are typically not static but 
are portrayed or manifested differently ^thin different contents and contexts. As one conception, belief, 
or set of understandings are explored in relation to others, implicit relationships and connections are 
revealed and conceptions are adjusted and changed as these become evident/expiidt to the individual. 
However, some individuals may not have a learning orientation or motivations which compel them to 
explore relationships anxxng understandings. Their understandings and epistemological franr^eworks may 
change slowly or appear static. 

Assessment of epistemological frameworks is difficult because one cannot look at epistenr)ologies 
through a single lens a see an individuars epistert r2k>gical framework in total through one attempt. In this 
study, individual profiles, representing a range of constructions in different contexts, were used to 
develop a typology of epistenx)logical framewaks. Participants' broad pedagogical understandings in 
relation to the typology and the development of epistemological frameworks are described in a later section 
of this paper. Reflectivity will be discussed next as a major contributing factor to developing deeper 
understandHigs of pedagogy. 

Teacher Reflectivity and Individual Learning Processes 
A reflective attitude is congruent with Schon's (1 988) conceptions of taacher reflectivity. It is 
best defined as an indviduars propensity to engage in reasoning and to seek deeper meanings (Siegel, 
1 989). In the broadest sense, reflectivity is pronDOted throu^ encouraging the active construction of 
meaning. The process may occur through such means as metacognition and self-reflection, (or simply 
reflection upon one's ideas) while the r^lt, we hope, is deeper understandngs and the development of 
one's persona! understandngs and eplstemologies. What one ultimately hopes is that reflectivity wiil 
become a lifelong habit and become integral to teaching and learning. While McDaniel and Lawrence (1 990) 
describe a reflective attitude as a major component influencing cognitive complexity, they do not explain 
how reflectivity Is developed nor how it can be encouraged by the instructional process. Basic to this study 
is seeing reflectivity as already present in individuals, in vastly varying degrees, and as either facilitated, 
mostly unaffected, or limited by instmction. A nun*er instructional methods facilitate greater 
reflectivity, such as joumals about developing conceptions of pedagogy (O'Loughlin, 1 990) a dscussing 
actual teaching experiences in the dassroom. In this study, the focus is on indviduals developing deeper 
understandngs throu^ constructing responses to authentk: teaching and learning tasks. The degree of 
authentidty of tasks is adjusted accordng to the partidpants' level of development. Pertinent to developing 
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reflectivity, according to this definition and In the context of instruction in teacher education programs, is 
the development of deeper learning approaches. 

Ent^^/histle and Nev^ble (EntiA^istle, 1 988) have fonmrfated a model which descril^es three nruijor 
learning approaches: surface, strategic, and deep. In their nxxlel, an individual's learning approach is 
infkaenced by predominant nr>otivations, processes, and outcomes. An individual with a surface approach is 
motivated by a concern for completion of the course and fear of failure. This indvkiual focuses on learning 
Isolated pieces of infonrwtion through mostly rote learning techniques. The outcome is either little or no 
understanding or a superficial level of understanding (substantial kno^edge of factual information and 
providing adequate descriptions). An Individual with a strategic approach is motivated by achievement of 
high grades and competing with others. This individual focuses on being successful (getting high grades) by 
using whatever means necessary. The outcome is a variable level of understanding dependng on what is 
required in a course or by the methods of assessment utilized. An individual with a deep approach is 
motivated by an interest in the subject matter and by vocational relevance (the intention is to reach 
personal understandings). There can be three outcomes: (a) a detailed knowledge of relevant facts 
(evidence) but little inlegration of evkience with broad principles, (b) ideas are related together based on 
relationships between ideas but are unsupported by evidence, and (c) evidence and facts are related to 
ideas (broad principles) and is used to develop arguments. 

Several pertinent points may be made about this model in general. First, the model indicates that 
making classroom instruction interesting and relevant and encouraging learners to search for personal 
understandings may also encourage learners to adopt deeper approaches to learning. It is difficult to expect 
students to be reflective without adopting this view of learning. Secondly, individuals* fear of failure and 
course completion needs to be attended to in order to expect rrKxe meaningful learning. Learners need safe 
atmospheres where they feel personal construction are valued and the instructor is there for both cognitive 
and emotional support. Third, the course structure (and ^assroom environment) and n>ethod of assessment 
interact with the way individuals approach learning and can limit or facilitate avenues for personal 
constructions. In essence, the method of assessment tells the learner ^at the instructor sees as important 
and therefore may affect how the learner approaches subject matter. As students encounter ideas they 
make meaning of tliese ideas by developing deeper understandings. This occurs by structuring and 
restructuring their own meanings about pedagogy and this process interfaces with and is infkjenced by 
instruction. How instructional processes and assessnr^ent can provide for these needed supports will be 
considered next. 

Broad Considerations for Instructional Processes 
How can instruction, and instructors, help individuals to become teachers ^^ho can not only cope but 
excel in the complex environment of the dassroom? A proactive approach in developing instructional 
experiences must be taken because the answer is certainly not to expect them to just assin)ilate an ever 
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grcHving body of infooratlon on teaching and learning^ expect them to becoa)e experts in subject matter 
content, while at the sanr>e Xkr\e overcoming years of a socialization process \vhich may not have prepared 
them well, essentially expecting them to team their practice through experience after being placed in a 
dassroom. Livingston and Borico (1 989) suggest that teacher preparation programs design activities 
explicitty to 'tielp novices develop and elaborate kno\Medge structures for teaching and pedagogical 
reasoning skills" (p. 39). However, this prescription may be too narrow. Kagan (1 990) states that 
cognitive views in general have failed to affect the nature of teacher preparation programs and teacher 
assessment. Cognitive approaches may have failed In part because broader considerations of the 
instructional prcx:ess and of the learner have not been taken into account. One must account for iTiore than 
cognitive aspects and include also the sodalizatkxi process In which cognitive growth is embedded (Goodad, 
1 990; Pintrich, 1 990; Veenman, 1 984). A reasonable solution is to treat the socialization process as one 
where teacher educatkxi encourages a cognitive apprenticeship and mentoring approach that aids and values 
the development of individual pedagogical constructions, epistemological frameworks, and personal growth 
and provides a community that supports and fosters individuals' commitment to and knovUedge about 
teaching and teaming. Planning fa instruction which facilitates preservice teachers development of broad 
and solid conceptual frameworks about the purposes of education, which has a dear conceptkxi of a basts in 
pedagogy, which accounts for teacNng and teaming from a holistic and dynamic view, and which seeks to 
develop common discourse would interact and support this type of socialization process. Additionally, it is 
important that teachers leam how to assess the entire range of their own students' understandings and set- 
up an environment which maximizes nvultiple levels of development. To do so means that teachers must gain 
experience and knowledge of these activities by being engaged themselves In sodalizatlon processes, 
developing understandings of how these processes are facilitated through tiie course/dassroom structures. 
Instructional activity, and nr^thods of assessnr^nt. A course/classroom structure should be viewed not 
only in terms of the subject matter organizatkxi and the activities but also as integral to the socialization 
process. 

Within the groups of individuals I have taught, as well as in the partidpants in this study, there is 
evident and broad variation, both in what these preservice teachers already know about teaching/learning 
and in Jteir learning approaches. It is important to start with what preservice teachers' already know and 
how they structure what they know in ader to promote development As is pronr>oted In Entwhistle and 
Newble's (1 989) nxxiei of learning approaches, a guiding conception in my instructk)n and in developing this 
study was to start with the preface that a leamer's nriotivations and learning approach are connected to the 
course i^tructure and the instructa's methods of assessment and evaluation. Recent calls for authentk: and 
alternative assessments that are systemically valid advocate the need for higher-order thinking and 
assessnnent as integral to instruction (Frederiksen, 1984; Frederiksen & Collins, 1989; Gardner, 1989; 
Linn, 1989; Pintrich, 1990; Resnick & Resnick, 1989; Wiggins, 1989). Course structures which emphasize 
ongoing, authentrc/attemative assessments, and open-ended tasks encourages ieamei^ to explore and 
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construct their own meanings. When these are integrated with the in >tructional and socialization processes 
nientloned (such as cognitive scaffolding and mentoring) then an environment for promoting deeper 
meanings of subject matter and concerns for personal growth has been optimized The tasks presented to 
participants in this study promote authentic activity in relation to preservice teachers' levels of 
development 

1 . The participants were engaged in continuous writing and planning over the semester. 

2. The participants were able to apply their knowledge of the subject matter and level of understancJng of 
the age group they >mII be teaching. 

3. The partkapants were able to use and make explicit the knowtedge and theories they already had about 
teaching and learning and integrate new knowledge and concepts. 

4. The tasks ask that the partfcipants integrate different ideas and concepts about teadiing and teaming. 

5. The completed works could be adapted or elalxrated upon for later use in the dassroom 

6. The nature of the tasks required individuals to put thenreelves in the role of a teacher when considering 
situations. 

In addition, several instructional processes were employed in the course to help facilitate 
participants' understanding. The participants were allowed to redo assignments after feecback to adjust and 
refine their ideas. This is sinr)ilar to what Shulman (1 989) calls including coaching in the assessment of 
teaching portfolios. Allowing assignments to be refined after feecback is more representative of the real 
worid of teaching where others may give advice, resources may be utilized, and in the sense that plans and 
ideas are never solidified and set, but are constantly changed, modified, and adapted. Similariy, examples 
or partial examples of task responses were often provided during instruction. Group dscussions, which 
were both planned and impromptu during classtime, also provided clarification or elaboration on 
misunderstood aspects of the assignments. These processes not only provide for cognitive support, but 
lessen indviduab fears that they have to understand completely the first time they attempt a task. 

DevelooinQ OoerhEnded Tasks 
Befmett (1 988) also emphasizes a focus on the nature and quality of dassroom tasks. To develop 
open-ended assigiments, sever?' considerations guided the choice of tasks and instructional strategies. The 
first consideration centered around task stmctures. in other words, how much structure is given by the 
task description itself and how much structure is the student asked to impose themselves, in previous work 
on cogiitive complexity (McDaniel & Lawrence, 1 9S0), a colleague and I had asked high school students to 
respond in writing to complex stimuli (videotapes) about cun-ent and past social issues. The task 
description was simply for the students to tell us their thoughts and ideas about these situations and to 
explain their thoughts as fully as possible, there was no cueing as to what they should focus on or any 
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criteria to which they must adhere in responding. In niost open-ended learning tasks, however, the 
structuring purposes of tasks are typically to guide and scaffold students, make materials intrinslcaliy 
interesting and challengng, and develop assignments which are open to nfieaningful individual constructions 
and which elicit certain types of knowledge or understandings. Since instructional activity, broadly defined, 
is the primary means throu^ which students learn, activity should serve as both impetus and scaffolding 
for individual growth. In general, however, task cues and prompts vary according to the type of 
information whk:h desired and the intended purpose. Totaly open-ended task prompts are useful in 
exploring individuals realm of understandings in a manner similar to open-ended, unstructured interviews. 

Decisions must be made about what types of knowledge are important to eBdt given the time 
constraints of a course, considerations for adequate assessment and feedback, and the demands placed on 
learners. The meaning individuals give to independent concepts is not rich in information about their 
pedagogical understandings although their understandings are easy to access and few co^rtive demands are 
placed upon the learners. More global tasks, such as longer term projects and papers, provide more 
information about pedagogical understandings and their ability to reason through situatkxis, but if too global 
are less useful in providing consistent feedback to leamers unless they are monitored in stages of project 
development. They are also difficult to score hdistically and difficulties may arise in promoting a 
developmental continuum. The tasks chosen for this study struck a balance in these many concerns. They 
were semi-structured and required students to plan for various leaming considerations and instructional 
activities. They constitute fairiy short tasks which could be completed in a page or so of writing 
(sometimes drawings and charts) but elicited a wide range of pedagogical understandings. 

In developing a method of assessment for this study, it was important to first determine how the 
participants conceptualize teaching and leaming, to ascertain the basic themes that permeate their 
pedagogical understandings, and to devek)p kieas about the quality and range of pedagogical understandings. 
The two sets of scoring rationales that were developed. Levels of Understanding of Pedagogical Themes 
(context g&neral) and Levels of Pedagogical Understandrig (context specific), take into account the 
variation of individual constmctions while reflecting participants* general depth of pedagogical 
understanding and the way understandings are integrated Since this is the first attempt at developing a 
means of assessment for pedagogical understanding, the use of these rationales in instructkxi is not fully 
expkxed. However, there is evidence that integrating these opert-ended tasks and this nr>ethod of . 
assessment could further an individuaPs understanding of pedagogy and help individuals develop 
epistemological frameworks. 



Method 



Participants 

The participants were 72 individuals. 1 5 males and 57 females, enrolled in two sections (section 1 . 
n « 40; section 2, n - 32) of an undergraduate introductory educational psychology course at Purdue 
University. This was a required course for 97% of the participants. Of these individuals, the ages ranged 
from 19 to 38 years, the nfiean age being 20.3 years. All participants were in their fourth semester of 
college or beyond (75% were in the second semester of their sophomore year). The distribution of 
participants stated grade level emphases upon graduation were as follows: preschool/ldndergarten (7%). 
elementary school (45%). junior/senior high school (41%), adult/unspedfied (7%). 

Overall Procedure 

The course included an emphasis on open-ended assignments and accepting the author as both 
instructor and researcher in the course, both of which were explained the first week of class and discussed 
with the students. The participants were neither recruited nor forced into participating, but were asked to 
participate as a part of this study. Although the course was a requirement for nrKJSt participants, the 
partfcipants could choose, within their scheduling limitatkxis. among seven sections of the course. The 
other five sections of the course emphasized ctosed-ended tasks and assessment methods as the major 
basis for grades (students were aware of the course dfferences). although some open-ended tasks and 
assessment methods were also employed. A few students joined the sectkxis involved in this study, two or 
three, because of the emphasis on open-ended nf>ethods and possibly one or two left to join other sections 
because of their emphasis on ctoseck^nded methods. 

Participants were informed at the beginning of the semester that permission to use their work 
dd not have to be granted until the end of the semester. In this way, partrcipants could hold back 
permission if they felt the study in anyway interfered with their learning experiences. Each participant 
completed a semester portfolio for the course. The portfolios contained responses to a variety of open- 
ended writing tasks ranging from considerations of specific pedagogical concepts and instructional 
strategies to extended projects. The portfdios also contained participants* evaluations of their 
progress and the course stoicture (at three dfferent points during the semester, although only the end 
of semester evaluations are presented here). The partidpanti also conpleted an inventory which 
assesses indviduals* learning processes. A set of 1 1 applied open-ended writing assignments were 
selected from the portfdios as the primary protocds examined in this study and are listed next (A 
brief description of each task is provided in Appendx A). 

PI: Convergent and Divergent Questions 
P2: Rde of Observational Learning 
P3: Individualizing Instruction 
P4: Self-Management 
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P5: Guiding Students in Strategies for Learning 

P6: Perception/Tecliniques for Cognitive & Learning Styles 

P7: Evaluating Open-Ended Problems 

P8: Worksheet on Evaluating Mass Media 

P9: Evaluating Student's Thinking Processes 

PI 0: Plan to Avoid Discipline Problems and Create a Positive Environment 
P1 1 : Plan to Individualize Instruction 



While participating in the course, individuals' work was assessed on the following five equally 
weighted criteria for lack of a more sophisticated assessment system: 

1 . Appropriateness for age and subject matter 

2. Adherence to guiddires, criteria, and/or education^ psychology concepts 

3. In relation to their other work 

4. Thorou^ness and integration of kieas 

5. Conceptualization or originality 

Each of these five criteria were listed on assignments with feedback. Feedback was also provided 
within the text of their assignments as well as in other discussions. More sophisticated nr>ethods of 
assessment were developed for these tasks after the course was completed. The construction of these 
rationales is discussed in the following sections. 

Preliminary Analysis of Participants' Responses to the Portfolio Tasks 
As an initial investigation of the content represented in the tasks, broad pedagogical themes were 
formulated through a content analysis of participants' responses across all tasks. The pedagogical themes 
which emerged were: 

1 . Teacher and Student Roles 

2. Educational Goals 

3. Role of Content and Conceptual Stnicture 

4. Qassroom Activities and Instructional Plans 

5. Evaluation and Diagnosis 

6. Nature of Thinking 

Participants' responses were then analyzed to devetop levels of pedagogical understanding within 
each theme. This was accomplished through initial sorting of responses into high and low levels, with high 
levels representing the most sophistk:ated understandings and low levels the most simplistic 
understandings of each theme, initial judgement of quality was based on the following continuums seen in 
participants' responses by two judges. 
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Continuums of pedaooaicai themes . 

1 . Teacher student roles and control orientation 

Teacher controlled and autoaatic environment vs shared control and demoaatic environment 
Teacher as disseminator of InfomTation vs facilitator of teaming 

Teacher dictated structure and rules vs student involved creation of structure and goveming 
Emphasis on dassroom management vs emphasis on students' growth 

2. Educational goals 

Discrete relationship vs holistic relationship between goals, activities, and perspective of teaming 
Goals are not purposeful or meaningful vs goab are purposeful and meaningful 

Goals limit students' choices, self-direction, and interaction vs goals encourage choice, self-direction, and 
interaction 

Goals are restricted to a limited purpose vs goals are developed to accomplish multiple purposes 

3. Role of content and conceptual structure 

Conceptually simplistic vs conceptual scaffolding and conceptually rich 

Content slwfie (few connections to other ideas) vs complex (facilitating many connections to other ideas) 
Content valued itself vs content valued as vehicle for growth 
Surface orientation vs conceptual orientation 

Organization is overiy specific (rigid) or va^e vs balanced in focus and structure (flexible and dynamic) 

4. Evaluation and diagnosis 

Basic skills vs multiple levels of skills and knowledge eva!*»ated 
Descriptive of classrooms vs causal analysis of situations (diagnosis) 
Leamers dchotomized vs individual needs and abilities seen as multifaceted 
Rule centered vs process centered assessments and evaluations 

5. Qassroom activities and instructional plans 

Activities are dosed and teacher defined vs open to students' indvidual interpretations and structuring 
Extremes in structuring vs well-defined, balanced structure of activities 
Independent, discrete activities vs interrelated, integrated activities 
Single focus vs multiple considerations in planning and structuring 

Activities are rigid and set at beginning of instruction vs activities are flexible and subject to change based 
on a number of factors 

ll-defined vs well-defined relation between teaming outcomes and purpose of activity 

6. Nature of thinking 

Convergent solutions expected vs divergent interpretations possible 
Knowledge acquisition vs knowtedge use 

Learning as passive redpient process (reproductive thinking ) vs learning as active construction 
(productive thinking) 

Thinking as stagnant vs thinking as dynamic and changing 

MdJevels were subsequently chosen and then levels two and four based on the degree and 
sophistication of the qualities shown in each theme. Using five samples in each levels descriptions were 
then developed around the generalizable qualities apparent in each level as related to the themes. A 
description of each of the five tevels within the six pedagogical themes (Le vels of Understandng of 
Pedagogical Tliemes) is presented in Appendix B. 
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Idealized descriptions of individuals' functioning at extremes of pedagogical understandng on all 
categories based on themes were developed and are presented next 

Integrated descriptions of extremes in pedaoooical understanding 
Level 1 : Lx)west level of pedagogical understandng. 

The teacher's rde is viewed mainly as the dsseminator of knowledge and the head of an 
autocratic dassroom. Rules are created and dictated by the teacher. The teacher has a reactive 
outlook on class problems. The student's rde is to obey rules as given, carry out the teacher's 
Instructions, receive information and produce or reproduce correct answers. Basic skills, the 
inherent value of content and facts, and dassroom management are major emphases. The 
dassroom activities and goals are stmctured and defined, though not well devdoped or 
conceived, by the teacher. Activities appear as extreme cases in structuring: vague and 
understructured or rigid, sterile and overiy structured. Rationales for the goals and purposes 
of instruction are generally omitted, simplistic, or cannot be inferred. The instructional plans 
focus on independent and dsaete lessons with a single focus or goal (usually of acquiring 
knowledge), or no apparent goal or focus. Plans are conceptually simplistic and concrete with a 
surface orientation to learning. Convergent answers are the outcome of instruction and 
evaluations are based on a rigid procedure or set of mles which define con*ect or appropriate 
answers, although the rdationship between the purpose of an activity and the learning outcome 
Is typically ilktefined. Student needs are defined in terms of simplistic categories and 
stereotypes and are not wel integrated into concepts of classroom interaction and learning. 

Level 5: Highest level of pedagogical understanding. 

The teacher's rde is viewed mainly as the facilitator of knowledge and as the head of a 
democratic classroom structure. The teacher accomplishes goals and sdves problems through 
proactive involvement. Many goals are co-created by the teacher and the student. The student*s 
rde is to partidpate in the group formulation of dassroom rules and consequences, partidpate 
in their own growth, and interact both cooperatively and independently. Thinking strategies, the 
inherent value of content as a vehicle for growth, and self-actualization are major emphases. 
The classroom activities and goals are initially structured and conceptually defined by the 
teacher to allow far subsequent student-imposed structuring, encourage cognitive processes 
and development, and acknowledge indvidual needs. Activities are structurally wdl-developed, 
dynamic, and allow for pdnts of convergence & divergence and connectkxi to other subject 
areas, concepts, and problem sdving skills. Rationales for the goals and purposes of instruction 
are wdl-devdoped, bdng conceived, supported and analyzed in terms of general goals of 
education, the objectives of the unit, and the characteristics of the students. The instructional 
plans focus on fessons with multiple goals and take into account the many complex facets of the 
students, the dassroom, and leaming with a constructivist orientation to understandng. Many 
instructional outcomes are taken into account and evaluatkxis are based on ntuttiple 
considerations. Student needs are described in terms of broad and spedfic sodal, emotional, 
moral, and cognitive concerns, construction of personal nr)eanings and are integ-ated into 
concepts of dassroom interaction and leaming. 



Only a small percent of the partidpants would fit into either of these extremes, however, the 
descriptions are useful in initially conceiving the boundaries for assessnwit and represent a dynamic and 
holistic view of pedagogical understandings and their interdependency. The use of the scoring rationales for 
Levels of Understandng of Pedagogical Themes were not further validated nor was a reliability study 
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conducted These themes are useful in discussing epistemological frameworks (as will be taken up in a later 
section) and in exploring the content represented in a group of tasks, but are too cumbersome (inv^ffk:ient to 
directly use in scoring tasks for each individual. Therefore, task specific scoring rationales were developed 
to score participants' responses (as is discussed in the next section). 

Construction of Levels of Pedagogical Understandha Scoring Rationales 
Since each portfolio task calls for an emphasis on one or nrwe particular educational concepts, 
processes, and/or an instructk)nal plan, each task also represents a somewhat unique enphasis on one or 
more pedagogical themes and combinatkxi of themes. To devetop task specifk: scoring ratk)nales, responses 
from all individuals in one class section were sorted into piles, representing five levels of quality, for each 
portfolio task. The process Wds similar to that previously described except that it was necessary to 
employ pertinent themes (and continuum descriptions) and make adjustn>ents around task specific 
considerations. From these levels, four representative samples of each level were chosen and each level 
was described. The result is 1 1 scoring rationales (Levels of Pedagogical Understandng), one for each 
portfolio task. A library of exemplars was developed that includes a description of each task, scoring 
rationales for each level, examples of responses, and scoring guidelines. The entire library is too extensive 
to present in this paper. Following are examples of task specific rationales and responses at extremes. 
Scoring rationales for Levels of Pedagogical Understanding are presented in Appendix C. 

Sample Scoring Rationale and Examples . 

Task PI 0: Ptan to Avoid Discipline Problems and Create a Positive Environment, Describe how 
your classroom will be set-up to create a positive learning environment and describe 3 methods 
or ideas you will use to prevent discipline problems 

Level 1 Scoring Rationale: Problems are viewed from a reactive stance with the errphasis being 
on classroom management procedures and/or behavior being controlled by the teacher. Self- 
management is typically discussed in temns of student behavior and is seen as good/bad, 
right/wrong, conipliant/noncompliant, or indicating a problem with not getting homework 
finished on time, asocial behavior, or being unorganized. Pedagogical concepts are dsaete and 
unrelated (disjoint) or only one particular concept is raised and discussed, Uttle explanation or 
understanding of the concept of self-fnanagement is relayed. 

Level 1 Example: I plan on using a disciplinary action based on a course by Lee Canter. I've 
never taken the course but IVe heard about tt and used it and it works very well. First of alt, 
when a student misbehaves, i.e„ talks out in class, mns down the hall, playing around instead 
of paying attention, or bothering other students, his/her name w8l be put on the board. For 
nxBt students, this is enough to make them behave because the whole dass loiows what a name 
on the board means. If the student continues to misbehave, a check will be put next to his/her 
name. This will cause the student to tose 5 minutes of recess. Again, for most students this can 
be detrimental. If the student still persists in misbehaving, another check will be put by the 
students name. This means the student will go out for recess but will do extra credit, which is 
assigned by the teacher. If the student does pooriy on the extra credit, it will affect his/her 
grade. Finally, if a student still does not settle down, a conference will be set up with the 
parents, the student, the principal, and the teacher, I would think this last action would occur 
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vefy rarely. Althcxjgh this type of disciplinary action does require the teacher to do a little, it 
isn't a lot. The teacher just puts a name and checks on the board He/she doesn't reprimand at 
the time of the misbehawig occurrence. 

Level 5 Scoring Rationale: SelfHnanagement is seen in broad terms of creating avenues for 
learning, self-awareness, conflict resolution, and creating cooperative goals. Self-management 
typically implies emphasis on student needs, learning to reason out problems, and nr»aximum 
student freedom, participation, and responsibility for the age group. The ideas relayed involve 
avenues open to evolving change and therefore may not appear detailed Pedagogical concepts 
are conceptually understood, integrated and related to classroom experiences and activities. 

Level 5 Example: I would have the students answer the follovMng questions: What makes a 
dassroom successful? They wouW then answer the question in essay form. I would then have a 
discussion based on vA\at was said in the papers. We wouki then compare and contrast my 
expectations with theirs. From this we vmII develop our classroom rules. We vmII then as a 
class cone up with punishments. We as a dass will then enforce the rules. If someone violates a 
rule then he/she will have to identify ^at they were doing to disrupt the dass and how that 
may affect the others. Only severe actions would call for my expdiing a student from the room 
(i.e., fighting, drugs). The day-to-day order and rules will be those from the dass. My role of 
authority will only come into action when needed. I would rather have them resolve their own 
conflicts. I would discuss with my students what my expectations are of them in the dassroom 
(i.e., to be patient and polite when sorreone dse is talking). Also, I would want them to address 
the issue in a debate. 



In the examples on creating a positive environment and avoiding discipline problems, we see 
that in the tower level example students are not active in the process except to know what the rules 
are and see that their behavior is in accord with the rules. The goals of dassroom management limit 
student choice, self-direction, and interaction, whereas, in the higher level response, the goals 
encourage student choice, sdf-direction, and interaction. 

The Level 1 response indicates only a set of rules for handling disdpline problems and totally 
disregards any enphasis on sdf-management. Furthemiore, no description or explanation is given for 
\^4^y the chosen method would work. Sonrie of the ideas expressed are disconcerting such as v^tly doing 
poody on extra credit work should lower a student's grade, that taking disdplinary action is the prime 
means of preventing discipline problems, that this system will be used for all misbehavior, and that the 
system is primarily used because it is easy. 

In contrast, the Levd 5 response focuses on the students as active participants in classroom 
governance. The teacher acts as a fadlitator who guides the students through problenvsdving 
activities to help them fomxilate their own rules and understand the importance of the rules in tenms of 
others. The teacher relegates his or her authority to allow the students opportunities for self- 
responsibility encouraging student choice, self<lirection, and interaction. 

These responses also illustrate how a task didts cf fferent understandings of pedagogical concepts. 
Typically, in lower level responses, few concepts are elicited and those that are elicited are narrowly 
perceived and have few connecttons to other pedagogical conceptk)ns. In higher level responses, ind viduals 
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tend to make cormectJons >Mth other concepts, concepts v^*lich are seen as a cmdal and integral part of 
stnjcturing the task and creating solutions. These conceptual connections are seen in the Level 5 response 
where the student's role of leaning to take responsibility and actively participate is an integral part of 
avdcing d sdpline problems and aeating a positive environment. Conceptkans about the teacher's and 
students* role in the classroom influences the way participants' smicture tmi ' of the tasks as do many of 
the other pedagogk^al themes. Samples of all levels of responses to tasks are provided in Appendix D. 

Interrater reliabilitv . 

Each portfolio task was scored by two trained raters. The author served as the first rater while 
another individual, with an education background, served as the second rater. Rater 2 was given a 
description of the original dass asslgnnf>ent, a description of each level, and four representative examples 
for each level. Scoring guidelines were discussed. The author scored all 1 1 of the portfolio tasks for all 
participants. The second rater scored five of the portfolio tasks for all partidpants except for those used 
as scoring examples. Initial interrater reliabilities on the five assignments were r « .77, .89, .90, .92, and 
.68 (N « 46, 45, 46, 46, & 45 respecti^v/ely). The reliabilities are thought to be lower for the first 
assignment because of the rater's initial inexperience with the rating system and, for the last assignment, 
because of a time lapse between ratings of the first four and the last assignment (one month). 

Rater 2 was asked to rerate assignments one and five after an interim period of two months and 
one month respectively and after further discussion about the application of the scoring rationales. 
Subsequent inteo'ater reliabilities on assignments one and four were r .91, and .89 respectively. Most 
disagreements resulted in differences of only one levd between rater 1 and rater 2. All differences were 
adjudicated with one hundred percent agreement before subsequent statistical analysis. 

Results 

This section describes the preliminary analyses of the scoring rationales for Levels of Pedagogical 
Understandng. Presented are tables describing the means and standard deviations obtained for ratings on 
each portfdio task and the correlatksns of ratings on each portfofio assignment with the total ratings. 
Ratings on the portfolio tasks during the first half of the semester are compared with the ratings from the 
second half of the semester (Tables are presented in Appendx E). Additionally, a content and descriptive 
analysis of partidpants' end of the semester self/course-evaluations, and a comparison of partidpants' 
ratings on Levels of Pedagogical Understandng vy^th scores on a measure of orientation to learning (The 
Inventory of Learning Processes, Schmeck, Ribich, & Ramanaiah, 1 977) • 

A comparison of means and standard deviatk)ns in Table 1 shews that some tasks appear more 
difficult than others. Portfolio tasks PI , P5, and P7 had the lowest means and standard deviations. These 
standard deviations indcate that there was less variance in partidpants' responses for these tasks than in 
other tasks. However, when looking at the means and standard deviations across tasks for all partidpants. 
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the means and standard deviations generally increased over trne. ft is assumed that the actual tasks dd not 
become easier but that participants* ability to structure the tasks and obtain hi£^er levels of understanding 
increased with time. Implicit in this view is that the inaeasing ability to structure open-ended tasks is 
concomitant \Mth cognitive restructuring and developing deeper understancings of and connectkns between 
pedagogical themes and beliefs. 

(insert Table 1 about here) 

The internal consistency of the portfolio tasks as a vMioie, as indicated by coefficient alpha was .85. 
This relatively high internal consistency suggests that there is some general quality ^ich is pervasive in 
the portfoib assignments and that each rating makes a strong contribution to the overall ratings. 

The portfolio tasks were partitioned by time period, representing those assignnients in the first 
half of the semester and the second half of the semester. The portfolios ^ich fell into the first half of the 
semester are those from PI - P5. Those in the second half of the semester encompassed P6 - P1 1 . 
Between these two sets of assignments there was approximately a four-week time period. Table 2 
describes the frequency of occurrence of different ranges in means for the first and second halves of the 
semester. During the second semester, the proportion of participants with mean portfolio scores between 1 
and 2 fell from approximately 1 2% to approxi: nately 6%. At the same time, the proportion of participants 
at the top levels (means from 4 to 5) rose from approximately 1 7% to approximately 33%. 

(insert Table 2 about here) 

The results of a T-test show a significant difference in the means of the portfolios from the first 
half and second half of the semester (t ■ 4.31 3, P < .0001 , df » 68). The shift in percentage of participants 
receiving higher ratings on pedagogk:al understanding NMth a significant difference in means (comparing 
first half and second half partitionings) suggests that over the semester participants may have inaeased 
their ability to structure open-eoded tasks and developed deeper understandings of pedagogy. Although this 
is presented as only preliminary evidence. 

Analysis of Self/Course-Evaluations 
At the end of the semester, participants were asked to ^e a self/course- evaluation and were 
simply given the instruction to "Write an evaluation of your progress in this course." Most evaluations 
ranged from one to two pages in length. An example of a self/course-evaluatkyi from one of the 
participants is presented next. 

When I first started out in this class I had no idea how to write a lesson plan nor did I know 
of many activities to use in a class. As I progressed through this class, I learned from the class 
dscussions and the readngs about how diikJren act and need to be treated. I was able to bring in my 
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experiences from other classes I had this semester as well as in the past to create new activities. 
Since the beginning of the semester, I feel Tve grown tremenckxisly in nrvy thought process involving 
education and psychology aimed at children. One thing that has helped me to grow are the (xrtfolios 
assigned. 

For the most part, the actual assignments were somewhat vague as to exactly what to do. 
This was frustrating at first, but t learned I could bring in my cm) ideas without having to stay 
within restricted limits. I truly thinl( this class has helped me expand my thinking on educational 
matters. 

In general, participants reported that they were uncomfortable or frustrated at first because the 
assignments were vague or not specific erxxigh. The tasks presented at the beginning of the semester were 
more difficult for participants because they were not facile in stmcturing open-ended problems and 
applying their ideas about pedagogy. A small percentage of participants were critical of the portfolio 
assignments and/or the course structure in general. These participants each felt there were some barriers 
to their progress. In most cases, except wtiere the participant cited external influences, the criticisms 
cited were the tack of concrete and specific tasks with set answers. 

The responses (N = 67), typically one to two pages in length, were collapsed into similar 
statements and grouped into five general categories; (a) teaching as a profession, (b) cognitive 
devetopment, (c) personal growth, (d) attribution of change, and (e) course in general. Table 3 presents an 
analysis of participants* responses which reflect participants' most salient thoughts about their progress 
and experiences over the semester. An overwhelming number of the responses were positive. To aid in 
understanding the nature of the responses a brief discussion concerning each major category is presented 
next. 

(insert Table 3 about here) 

Teaching as a Profession . Partk:ipants ^o said they had an increase in respect and knowledge about 
teaching, 1 9 percent of the total responding, relayed that they have learned that teaching is a challenging 
profession v^liich requires many skills and different types of knowledge and/or that they have a better 
understanding of what it is like to be a teacher. A portion of the partk:ipants, 1 1 percent, stated that over 
the semester they had become more committed to teaching and/or realized that being a teacher is 
something they want and need to do. A much larger portion of the participants, 30 percent, stated that 
over the semester they had leamed some generalizable capability (such as aeativity) necessary to 
teaching or felt that they are now better prepared to teach. A large portfon of the participants, 46 percent, 
specified a better understanding of important concepts or obtained valuable skills such as nfK>tivating 
students, classroom management, organizing classroom settings, meeting indi\idual needs, adapting 
diverse methods and strategies, applying theories and psychological concepts to teaching, creating lessons, 
developing activities, and practical knowledge of teaching. No negative responses were given in this 
category. 
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Cognitive Devdopment , Statements coded into this category alluded to the development of reasoning, 
acquiring broad concepts, or seeing the connection among knowledge In different domains. A large percent 
of the participants, 32 percent, stated that they had learned how to develop their ideas, to support their 
ideas ^h evidence, and had expanded their knovs4edge base. The same percent of the participants stated 
that they became better at stmcturing open-ended situations, even though a large majuity said that the 
experience was initially frustrating. An even larger percent of the participants, 43 percenr, made 
statements that the knov^^edge they teamed had direct connections or relevance to their future teaching by 
stating that either the ideas they have developed would be used in their classrooms or that the knov^edye 
learned was beneficial or pertinent to their career as a teacher. Staten^ents conceming the knowledge 
gained in this course being of relevance to other courses and experiences or the use of previous or general 
knowledge to participating in this course and structuring open-ended assignments were dted by 1 6 percent 
of the participants. Three percent of the participants made a general statement saying they riid not learn 
very much. 

Personal Growth . Statements relating to an increase in self-esteem and confidence, 27 percent of the 
participants, include bang more comfortable at participating, nrwe secure about one's own ideis, feeling 
more adequately prepared for being in the classroom (this was taken as being more emotionally prepared), 
nrx3re confident in handing problems (or whatever else comes abng), more competent as an educator, and 
feeling thQt one's ideas were accepted. Learning how to express one's own ideas, knowing where to start on 
open-ended problems, increased self-discipline, and learning from experiences were all statements made 
about self-direction. Such statements were made by 24 percent of the partk:ipants. A large percent of the 
participants, 43 percent, stated that their self-awareness inaeased, many adding examples of personal 
progress or personal discoveries. 

Attribution of Change . Direct attribution of change because of the open-«nded tasks was made by 41 
percent of the participants who generally stated that the assigiments were stimulating and made them think 
about different ideas or situations. Three percent said that the assignments were too vague. Similariy, 38 
percent of the participants stated that their progress could be directly attributed to the course in general. 
The statements were either general in nature stating a benefit gained from the dass or that the course was 
beneficial because of different aspects such as the variety of topics, the open method of teaching and 
learning, and leaming more than book facts. A majority of the participants, 65 percent, stated that they 
put much time and effort into into the dass and assignnrients and it was worth it because of personal growth 
or an increase in the quality of their work. Eight percent said that it was too much work or that because of 
the anrxxjnt of effort they put in they shouW receive a high g-ade for the course. Based on partidpants 
comments about their peers and the instructor, 22 percent said it was a positive experience because th^ 
learned from others* ideas and discussions, feedback was benefidal, or through interacting they learned 



how to communicate with their peers. Three percent said they could not decide what the instructor's goals 
were. 

Course in General. Comnwits about the course in general included statements from 27 percent of the 
participants on the positive aspt^ of the course staicture. These conrvrwnts inducted liking the course 
because it was informal, the dass members were active partidpants with a great amount of input, the 
presentation of ideas were unique, that it was a positive experience to be in a class with others who would 
be teaching a wide range of subject and age groups, and that the dass was set-up as individualized work 
and personal discovery. Bght percent of the partidpants said either the course was too much work for 
three credits, the book was slighted, the dass was drected toward a lot of writing assignments and 
writing ability, there was a lot of time spent on theories that weren't related to education or practical 
uses, and/or the course dealt with things that should; t be covered by a psychology course. Many 
partidpants made comparisons with other courses or experiences (39 percent) saying that this experience 
was more benefidal than other dasses or assignments they have had A large percent of the partidpants, 
46 pereerit, stated general positive feelings about the course. Such comments induded being glad to have 
this opportunity, the course evaluations were fair, enjoying the dass, and that the class was interesting. 
Five percent of the partidpants reported negative feelings about the course stating that they ddn't learn 
very much, they should not have been required to take this course, and/or the that course was frustrating 
for them. 

The partidpants' evaluations provide some support for the assumption that the dfferences in 
means over the semester reflect cognitive growth and not decreasing difficulty of tasks. Because the 
concepts in each successive portfdio build upon one another, it would be highly unlikely that even the same 
level of integration and complexity could be maintained unless some reorganization of participants' 
conceptual networks and frames of reference also occurred. 

The next section presents an analysis of the relationship between ratings on the Levels of 
Pedagogical Understanding and scaes on The Inventory of Leaming Processes. 

Comparison of Levels of Pedaaodcal Understanding and Learning Orientation 
The purpose of this analysis was to examine the relationships between ratings on Levels of 
Pedagogical Understandng and an important related construct, leaming orientation. The Inventory of 
Leaming Processes (Schmeck, Ribich, & Ramanaiah, 1977) is a 62 item, true-false, self-report inventory 
of conceptual and behavioral processes indviduils engage in when leaming new material. The inventory 
contains four scales briefly described next. 
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1 . Synthesis-analysis (1 8 itenis, later relabeled deep processing). Measures a meaningful orientation as 
opposed t 1^ superficial orient )n to processing infomiation. The internal consistency is reported as .52 
and a test-retest reliability as .88 (Schnr)eck & Ribich, 1 978). 

2. Elaborative processing (14 items). Measures an orientation to engage in elaborative processing 
strategies versus verbatim processing strategies. The internal consistency is reported as .67 and test- 
retest reliability as .80 (Schmeck & Ribich, 1 978). 

3. Fact retention (7 items). Measures attention to detail in processing as opposed to attention to 
generalities. The internal consistency Is reported as .58 and test-retest reliability as .79 (Schmeck & 
Ribich, 1978). 

4. Study methods (23 items). Measures the use of organized and persistent study habits and methods 
versus other methods such as repetitive practices, cramming, and rote drill. The intemal consistency is 
reported as .74 and test-retest reliability as .83 (Schnneck & Ribich, 1978). 

Participants completed the inventory three weeks into the semester. Significant con'elations were 
found with the mean ratings on Levels of Pedagogical Understanding (total semester, N « 68; first hair, N ^ 
64; second half, N = 67) and scores on most subscales of the Inventory cf Learning Processes. This pattern 
of correlation coefficients supports th xpectation that indiMduals who already had thoughtful and 
reflective approaches to learning woulu perform at higher levels on the portfolio tasks. As shown in Table 
6, no significant correlations were found with ratings on Levels of Pedagogical Understanding and the 
scores on the fact retention subscale, a scale measuring efforts to acquire and remember isolated 
information. Significant positive correlations (r « .42, p < .001 with total ratings) were obtained with 
study metfiods, a scale stressing organizational versus rote processes in learning. Perhaps the most 
meaningful correlations are those between Levels of Pedagogical Understanding and the analysis/synthesis 
subscale (r » .42, p < .001 with total ratings). Schmeck later renamed tiiis subscale deep processing to 
more fully communicate its focus on the extent to vsliich students critically evaluate, conceptualize, 
organize, and compare and contrast the ir^cTnation they study. These are the types of processes which 
should contribute to high levels of per^agogical Uk>derstanding in the portfolio tasks. 

(insert Table 4 about here) 

Responses on self/course-evakiations at the end of the semester indicate that many of the 
participants saw a change in their own orientations to learning (to a vnore corKeptual and deeper 
orientation) and described how their thought processes had devek)ped over the semester. How orientation 
to learning and cognition interact to influence different individuals* learning should be more fully explored in 
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future studies. Had the instalment been administered again at the end of the semester a positive change in 
learning orientation may liave been discemat>le and correlations may have been higher between scores on 
this instrument and the second half of the sennester partitionings fa Levels of Pedagogical Understanding. 

Exploratory Study: Construction of A TvpoIoqv of EpSstemdoqical Frameworl<s 
The following discussion illustrates how participants' responses reflect their conceptual 
development and their formation of epistemological frameworks. In general, an individual's epistemological 
framework represents predominant understandings of pedagogy, how pedagogical themes interact and are 
understood, and how these understand ngs appear when interpreted as a vMde. 

Typology of Epistemdoaical Frarmeworks 

A typology of epistemological frameworks describes four levels of epistemological development and 
the conceptual processes which function in each level. These levels are simplistic, disparate, fluctuating, 
and developed epistemological frameworks. These were developed through an in-depth analysis of 
individual's work and their ratings on Levels of Pedagogical Understanding. While these descriptions were 
developed on ratings on Levels of Pedagogcal Understanding over the semester, partidpants ratings often 
indicate that they may fall into different descriptions from one half of the semster to the next. For instance 
ratings for some individuals' work for the first half of the senriester indicates a sinnplistic epistemological 
framework whereas for the second semester ratings fall in the mid or upper levels and indicate other 
epistemological frameworks have been fomied Likewise, some participants remained fairiy stable over the 
semester. 

Simplistic epistenrx)logical frameworks (approximately 1 6 percent of the participants). These 
individuals have pedagogical understandings which span the lower three Levels of Pedagogical Understanding 
(levels 1, 2, & 3) with means of 1.4 - 2.3 on all ratings. The individual constructs the entire classroom 
environment around only a few ideas and the kteas are limited, based on a restricted view and a small 
number of classroom considerations. The conceptual networic related to any given major pedagogical 
concept, such as the teacher's role in the leaming process, is more discretely defined and interpreted, 
based on only a few nan-ow assumptions and concepts/ideas. Therefore, ideas are difficult to elaborate 
upon or connect to other ideas. These individuals wil interpret or make nr)eaning of the major pedagogical 
concepts as a set. For example, it is typically assumed or believed that the teacher's job is to control 
students' behavior, the students' role is to be good and wori< hard toward what they are told. A strong 
belief or conception that the classroom shouW be authoritarian helps define leaming as the passive 
acquisitk)n of content and the students' rde as passive. Activities are used as an avenue to learn content, 
content being seen as important in and of itself instead of as a vehtele for student growth. Individual 
differences may translate into easy dichotomies such as fast/slow leamers and instmctional strategies for 
aiding development into less/more practice or faster/slower pacing. Assessment is seen as giving grades 
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and rs based on whether or not the answers given by students are oyrect The undarlytng assumption is 
that content is relatively set with correct answers and set ways of learning those answers. A set of 
pedagogical conceptions nrtay be sustained because the ideas are in congruence with one another. 

Disparate eoistemoloQical frameworks (approximately 28 percent of the participants). These 
individuals hold conceptions which span all or at least four Levels of Pedagogical Understanding (levels 1-5 
or levels 1-4) with means of 2.5 - 3.4 on all ratings. It is probable that these individuals more fully 
understand some aspects of pedagogy or see some educational goals, concepts, or beliefs as important, 
such as meaningful learning, but not others, such as a caring conrmunity. Individuals may struggle to 
incorporate their ideas about thinldng and learning ^th wliat they view as ciassrco^n nnanagement. The 
concept of the teacher/students' roles are often disjoint from social roles and may not be seen as 
connected (or it is not understood how they are connected) to cognitive growth. The teacher may be 
oriented toward a denxxratic environment, but has some difficulties in gasping the ideas of students being 
in control of their own learning (developing the co^itive capabilities behind goal setting and self* 
nrxxiitoring) and teaming from interaction. In this case, the teacher's role rt nains custodial, as the person 
who knows best what is needed, limiting the full realization of student sdf-responsibilityand shared 
decisiorhmaking. Likewise, an individual may have different ideals/ideas about classrooms which limit 
other ideas. 

Fluctuating epistenrxjIoQical frameworks (approximately 45 percent of participants). These 
individuals may have understandings whk:h span middle Levds of Pedagogical Understanding (levels 2, 3, & 
4, although they may occasionally have a level 1 or level 5 rating) with means of 2.6 - 4.2 on all ratings. A 
disparity in pedagogical concepts is not evident. Independent conceptions are not yet solidified or robust, at 
tinnes they are nnanifested more simplisticalty and at other times wore complexiy. This does not occur 
because of a large disparity among conceptkxis and beliefs, but because their developing conceptions are 
fragile and in a state of flux. The complexity and interrelationships of conceptksns are evident w^en the 
context of the tasks to vM\ch they are applied is varied. As an example, an individual may manifest 
conceptions about individual differences in quite different ways, or have understandings of the same 
concept falling at cf fferent tevels of complexity. When the task caKs for developing divergent questions and 
criteria to evaluate responses to these questions, conceptions about individual differences are manifested 
quite cffferently than w4ien the task calls for designing an ambiguous stimulus, discussing how students 
would perceive the situation presented in different ways (such as how students may interpret the meaning 
of a poem, a visual depiction of a human encounter, a song, or a coltection of historical accounts of the 
same event), and planning for activities which encourage higherHevd thinking. Each of these tasks reveals 
how a conplex concept, such as individual differences, is integrated \Afth other ideas. SoikJ connections 
between individual differences and how students construct meanings nrwy be lacking if criteria for 
assessing students' responses are not based on meaning-making as a dynamk: and evolving concept. 
Likewise, an indlvkiual nr^ay identify and discuss student development (both cognitive and social) as an 



innportant ind vidual dfference, but the concept of the development of meaning renr»ins relatively vague 
and unrelated to gross student development. 

Developed eoistenroloQlcal frameworks (approximately 1 2 percent of ail participants). These 
individuals* responses typically span only two Levels of Pedagogical Understandng (levels 4 & 5, although 
they may occasionally have a level 3 rating) with means of 4.3 - 4.8 on all ratings. Individuals who 
consistently function at hi^ levels of understandng manifest deeper understandngs of pedago^cal 
concepts and their interrelationships. Pedagogical understandngs of student self-fesponsibility, indvidual 
differences and needs, student-teacher roles in the classroom, learning as active construction, activity and 
content (subject matter) structures, and assessment methods are integrated VMth one another. For 
instance, actitivties are often planned to specifically elicit indvidual constructions of important themes, 
areas of learning, or subject matter content. Scaffoldng to enhance student learning is included as both a 
learning activity and as a dacpostic assessment of learning. Leaming takes place by teacher-student and 
student-student costructuring of ideas and goals. The rde of the teacher and students is intimately 
connected to students formulating some of their own structure for leaming (problems, solutions, and means 
of study) as well as facilitating a positive environment for other types of growth. Many types of growth 
are planned into activity structures and subject themes, often resulting in interdsciplinary instruction. 
Ideas about pedagogy inform and aid the indvidual in developing dynarrwc leaming situatkxis that take into 
account a number of dassroom consideratkyis based on broad pedagogical understandngs, implicitly or 
explicitly stated goals of education, and what is seen as purposeful in teaching and leaming. More specific 
considerations for indvidual students' needs, for group leaming, subject matter, etc. are connected to 
these understandngs. 

Indvidual profiles, in-<iepth descriptions, of indviduals simplistic and developed epistemdogical 
frameworks in relation to leaming approach were also described for this study, but due to the length can 
not be presented here as supporting evidence for the proposed typology. Work underway includes a 
description of indvidual profiles of the mid-levels of epistenrxdogical development The profiles depict the 
indvidual variation in the way meaning is constnjcted in relation to classroom contexts and how 
epistemological frameworks and the interrelationships of pedagogical concepts are formed. A rich an^ay of 
indviduals' work (includng other work not presented in this paper) is used as evidence of how they view 
their role in the classroom, what their educational goals emphasize, how their pedagogical understandngs 
affect curriculum developnrient and instructional plans, the leaming which will take place in their 
classrooms, and how they will assess their o\mi students' progress. The indvidual profile are dscussed in 
li^t of leaming approaches (based on Entwhistle and Newble's descriptions of surface, strategic, and deep 
orientations to leaming) and indviduals* unique strategies for encouraging classroom leaming and develop 
curriculum and actitivities around subject matter. 
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Disa *>sion and Implications of the Study 

Participants' responses to the tastes developed for this study indicate how they might plan and solve 
instructional and managerial problems in a classroom context utilizing pedagogical understandngs. The 
tasl<5 encourage individuals to malce explicit their ideas about ^at is important for students to leam and to 
structure the subject matter in meaningful and appropriate ways for the age group they will be teaching. 
The tasks also challenge individuals to reflect and think about their own teaching strategies and how they 
conceptualize tlie many different facets of classrooms. 

The development of pedagogcal understanding is faditated when individuals have deeper 
approaches to learning, ^en they move to complex approaches in the way knowledge is perceived, 
organized, and integrated, and v\*ien they develop cognitive frameworics for interpreting their experiences 
and knowledge. The results of this study support these contentions through the analyses of the participants' 
task constructions. They are also supported through comparisons vAxh self/course evaluations, scores on 
learning processes, and a discussion about a typok)gy for educational epistemdogies. 

A notable influence on pedagogical understanding is an individual's orientation to learning, their 
consistent propensity to seek meaningful learning. The results of this study indicate that those individuals 
who do not adopt a reflective stance on learning (or, 2nak)gousty, a deeper orientation to learning) may 
have difficulty in furthering their conceptions and understandings of how their students will leam and 
develop. It seems necessary that if individuals are to become competent teachers they must also engage in 
the same processes that we would want them to encourage in their student'^. Rntrich (1 990) discusses the 
need for research to provide a comprehensive, sophisticated, and dynamic view of teachers which reflects 
an integration of motivatkxi and cognition. Reflective thinking is an important concern to teacher education 
because it seems likely that an individual would have difficulty in developing a reflective stance and 
therefore in developing deeper pedagogical conceptions once a practicing teacher. 

Participants' self-e/aluatkxis support contentions that the tasks encourage development of 
reasoning and of pedagogical understandings. In additk)n, these responses ir>dk:ate other important 
influences on the participants' growth. Particulariy important was the findng that personal growth is 
affected and tiiat a form of empowenncnt conr^ from examining and understandng one's own 
epistemological frameworks and ideas about pedagogy, in this study, participants' confidence about their 
worth and capabilities increased as their ideas became more fully developed. Participants clearty 
expressed important aspects of this process and the interrelatedness of knowledge, personal variables, and 
cmpovverment. Many of the partkapants' responses indkated that they now felt ready to take on the 
dfUcult chalSenges of teacNng or that they fett more capable because they have learned more about 
teaching. These statements were coupled ^th other statements about learning to develop or juTtify ideas, 
feeling more adequately prepared to teach because knovs4edge was gained, and/or feeling more secure about 
their own ideas. 
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Richardson (1 990) advocates that teachers be in control of change, and 'teachers understand and 
be held accountabJe for the pedagogical and moral implications of their decisions" (p. 1 3). Teacher 
educators, involved in evaluating preservice teachers and practicing teachers should help them understand 
the Implications of their decisions. Throu^ developing pedagogical understandings, individuals develop an 
infooDed discourse about teaching, learning, and worthwiMIe change in education. They also appear to gain a 
voice through developing competence and feeling personaly capable. This sort of inftxmed dscourse and 
voice can help to facilitate meaningful discissions about pedagogy within the educational community. 
Without thiis facilitation, it would be difficult to expect teachers to understand or voice the implications of 
their decisions. 

Implications and Applications of the Scoring Rationales 
The direct implications from participants* responses indicate several areas to focus on when 
encouraging the development of understandings. A main concern is for individuals to develop conceptions 
about their students' learning as a process of active construction. In general, participants who understand 
learning as active construction developed plans which encourage students to engage in: (a) applying their 
knowledge, (b) displaying conceptual understandings of key ideas, (c) relating knowledge to broad concepts, 
and (d) forming conceptual networks. Participants who exhibited higher-levels of understanding also 
incorporated the following into their plans: 

1 . interdisciplinary instruction 

2. an environment where many tools and resources are available to students, students learn to use the 
tools and resources and choose what they need 

3. evidence of their role as facilitators and scaffolders 

4. assessment as an integral and ongoing part of instruction 

5. considerations for many forms of student development including social, moral, physical, emotional, and 
cognitive 

6. multifaceted/multilevel learning experiences related to educational goals 

7. teacher-student shared control, governing, and structuring of learning, teacher has control at certain 
necessary points 

It is proposed that these characteristic of higher-level understanding could become teaming goals of 
teacher education courses. Whether all individuals can reach acceptable levels of competence cannot be 
cannot be fully addressed in this study. It seems obvious that if an individual obtains only lower-levels of 
understandng by the end of their teacher education program, these levels of competence would not be 
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acceptable. However, these considerations should be explored before radical restnjcturing of teacher 
education is pronx>ted such as having higher order and basic skills teachers as in Hannawa/s (1 992) 
proposal. This study was conducted over only one semester, however, the results were positive even in 
this short amount of time. It may take more time for some individuals. It is possible that a vwy small 
number of individuals would not ever develop deeper pedagogcal understandngs. This appears to be the 
case for only a small number of the participants in this study. Given the time restrictions in this study and 
the fact that most participants were only in their sophomore year of study, the outlook is hopeful for high 
success rates using similar methods. Furthermore, there is some initial resistance to and fear of open- 
ended assignments, but these feeings subside as individuals become assured that they are capable of 
thinking through problems and situations. 

Implications for Future Research 

A concem of this study is that the scoring rationales and method of assessment reflect and promote 
a devetopmental continuum in teacher education and in pedagogical understanding. Since the scoring 
rationales are based on levels of pedagogical understanding, they already measure a continuum of 
devetopment^ Future research may show how they could specifically be utilized in instructkxi. Other 
aspects of promoting a developmental continuum include their overiap with actual teaching practrce. 

The scoring rationales for Levels of Pedagogical Understanding have potential applrcations to other 
tasks. Much needed are tasks whk:h may indude opportunities for individuals to develop their beliefs in 
relation to situations more heavily emphasizing other pertinent classroom issues. Additional tasks could be 
developed to further individuals' understandngs about how cooperative learning activities could be planned, 
how in? ojctkxi might encourage sockxnoral and emotk)nal development in students, 2nd about the workings 
of social interaction in the classroom in relation to teacher and student roles. 

The task descriptions need to be refined. In many cases, smaS changes in the descriptions will 
affect how the student perceives and structures the tasks. For instance, further examination is needed 
about requirements stated in the tasks. One requirement which may be important to include is that 
individuals justify their plans, or include a rationale. Some of the tasks stated these requirements while 
others did not. The tasks which explicitly asked for rationales force individuals to think through their 
reasoning. Often, vA\en the requirements do not ask for a rationale, no explanations are given or the task is 
not as fully explored. Other aspects of how the wordng of tasks affect responses shoiid also be a focus of 
future research. 

As students progress, wore complex tasks would be appropriate, tasks whidi require more 
extensive development, and tasks which require the synthesis of broad concepts. Indviduals should 
probably also be required to construct nxre extensive pictures of the learning experiences they will 
promote in the classroom espedally as related to how students nuiy conceptualize the subject/ideas at 
hand, how instruction will interface with students* capabilities, how teachers provide scaffolding, how to 
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inteorate cSfferent domains of knowtedge and subject niatter, and how to ware fully assess the range of 
student learning. 

The rationales for Levels of Pedagogical Understanding may be utilized to evaluate the acquisition 
and appScation of concepts, especially broad conceptions of teaching and leaming and how these are 
organized into a set of epistemdogical beliefs about education. If this method were used as a part of the 
evaluation system, preservice teachers could evaluate their ovmi beliefs and develop their thoughts while 
having a criterion standard with which to compare their conceptions. Research on the assessment of 
competent thinking as an integral part of the teaching and leaming process is dscussed by NSckerson 
(1 989). Instituting this method of evaluation may also involve developing a system where individuals can 
strive for the next higher level of understanding. In Vygotsky's (1 978) temis, the instructor would provide 
scaffolding for the individual based on their level of understanding. In tum, individuals would leam how to 
provide this type of leaming environment for their future students. Furthermore, incSviduals would leam 
the rationale by applying it to spedfic subject nwtters, not only wouW subject specific knowledge and 
general processes in individuals be assessed, but it could allow these individuals the chance to begin 
developing instaiction and evaluatkxi systems to use with their own students. These projected outcomes 
are within the realm of possibilities with further research. 

Most cun-ent thought is that the teachers should be the assessors and gatherers of infomnation 
about students, this in tum should encourage teachers to become problen>solvers and c^amic evaluators, 
encouraging reflectivity in themselves. Frederiksen & Collins (1989) see particular benefits to employing a 
library of exemplars as a set of "case studies", showing students the nature of expert performance and 
helping them to "develop a keen sense of standards and critical judgement" (p. 30). The library of 
exemplars collected for this study could be used in such a manner with individual preservice teachers, for 
group discussions, or to train instructors in teacher education programs. 

Future research on assessing pedagogical understanding in conjunction vwth teacher development, 
including continuing development during practice, would be valuable. Rationales similar to the Levels of 
Pedagogical Understanding could be useful in discussing and assessing vkJeotapes in teacher training or in 
evaluating and providng constructive feedback to preservice teachers during student teaching. An overall 
evaluation procedure using the portfolio assignments, or adapting other tasks to the scoring rationale, in 
combination with graduated teaching experiences in classrooms woukJ be nrxjst benefidal in encouraging 
preservice teachers* development. During the course of a preservice teacher's underg-aduate education, a 
portfolio including many dfferent indicators of the individual's level of understanding should follow them 
and record their progress. Results from this study also indicate that utilizing other instruments in this 
assessment would be valuable. The instalments utilized in this study to measure cognitive complexity, 
adult intellectual and ethical development, and orientation to leaming would provide insights into the 
process of teacher developnient if utilized in conjunction with other indicators of development and over 
longer periods of time. 
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if an evaluation system for preservice teachers included measuring their level of pedagogical 
understarKings in educational contexts and these were implemented based on individual developnnent, 
increasing levels of understanding, and evolving epistemoiogical beliefs, it would be an empowering 
situation for individuals. It would also be a huge step toward preparing them for the complexities of the 
classroom. Without an integrated frame of reference the teacher will probably be ill-prepared to solve 
problems or will feel overwhelmed when faced \Mth the many complexities of the dassroora Using this 
method in preservice education 

within a limited context also allows individuals a safe environment to explore and develop their ideas. 

There is also evidence from the participants' self-evaluations that incorporating the methods 
employed in this study (with refinement) into classroom instruction would not only encourage self- 
reflection and self-regulation, but also would have positive effects related to other areas of personal 
growth such r<s Kicreases in self-esteem, confidence, and self-awareness. As Richardson (1 990) states 
that "...teacher empowemnent does not occur without reflection and the development to express 
justifications" (p. 1 6). 
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Appendix A: Brief Descriptions of Tasks 



(Note: actual descriptions often included extensive discussions of criteria, examples, and explanations to 
provide guidance and scaffoidng) 

PI ; ConN^rqent and Dtveraent Questions 

Develop nieaningful convergent and divergent questions covering one topic and state what the purpose of 
each question is in learning* 

P2: Rde of Observational Learning 

Develop a homevwk sheet that your students can use to evaluate some aspect of the rDeda. Students should 
learn from the evaluation and team something about the subject matter. Discuss what you would do with 
their results* 

P3: Individualizing Instruction 

Develop a short set of instoiction that can be used by a student to learn independently. Explain how this 
instruction would fit into the other dassroom assignnDents and the learning environment. 

P4: Self-Management 

Develop a plan for self-management for one student. The idea is that the student is gaining control over 
their teaming, you are there as a mentor and facilitator! 

P5: Guiding Students in Strategies for Leamina 

Devise a set of questions/statements to guide students in using strategies which benefit their teaming. 
P6: Perceotion/Technigues for Cognitive & Leaming Styles 

For this assigiment, pick out a stimulus which will be presented to your students which is open to 
interpretations. Describe the different responses you would get to this stimulus. Explain what the next 
step(s) in your instruction will be. 

P7: Evaluating Open-Ended Problems 

De\ise the criteria or method for evaluating student responses to a divergent question and provide a one 
paragraph rationale for your criteria* 

P8: Worksheet on Evaluating Mass Media 

Using the exercise you completed on observational leaming devise at guiding questions which encourage 
problem solving and critical thinking. 

P9: Evaluating Student's Thinking Processes 

Design a rating sheet to evaluate student thinking processes. Explain in enoug^i detail that you are showing 
how the criteria relate to your purposes. 

PI 0: Plan to Avoid Discipline Problenrg and Create a Positive Environment 

Describe how your classroom will be set-up to create a positive leaming environment and describe what 
nnethods/ideas you will use to prevent discipline problenr^. 

P1 1 : Plan to Individualize Instruction 

Explain how you would tailor your goals and objectives to fit the needs, interests, and abilities of different 
students. Describe or show what is different in the materials, your expectations, and how your plans will 
be implemented 
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Appendix C: Rationales for Levels of Pedagogical Understandng 



PI - Convergent and Divergent Questions 

Level 1 : The questions, when seen together, do not form a conceptual whole and may not be related except po«bly by 
subject area. The majority of the convergent and dh/ergent questions lack conceptual breadth and are simplistic, I.e., the 
dr/ergent c^estion may elicit only a few wads or even a one word answer v^ich may even be known and may only be 
open to limited interpretatkxis. 

Level 2: Most of the questions are lacking conceptual breadth and are often phrased simplistically, suting an obvkxis 
purpose or lacking a purpose. They may however include some conceptual breadth to the questions, or state a meaning- 
ful purpose. The questions typically fit together in the sense that they all cover the same topic, but are otherwise only 
vaguely conceptually related Some explanation may be given for the purpose behind the questions. 

Level 3: Questions are all related conceptually to the same theme or topic. Attempts are made at identifying and 
explaining the purpose behind the questions. Most questions are conceptually missing interim links even if related to a 
particular topic, however some of the questions do appear to beconceptually linked. 

Level 4: There Is an overal topic or theme or purpose. Convergent questions are formed on a more conceptual basts 
instead of memorized answers. As a whole, the assignment seems meaningful and conceptually structured Questk)ns 
lead to more understanding of the topic as a whole and purposes may also seem mae broad than at the previous level 
but are still vaguely defined in some cases. 

Level 5: There is an overal theme or purpose. Convergent questions are formed on a more oonoeptual basts as in the 
previous level and they are conceptually interrelated to dvergent questions, one questkyt (of either type) is used as a 
lead-m to the next question, or questions bring up different elements (subthemes) Important to the theme (there are 
possibty other organizational/structural characteristics which may be manifested in other assignments at this 
level). Often, explanations are given for why one question is used in a particular point in theinstmction and what other 
aspects of Instmction or method would be implemented upon use. 

P2 - Rde of Observational Learning 

Level 1 : Observation required of the students are minimal and do not seem to fit the assignment. Smplistic blanket 
assertions may be made about subsequent classroom activities (such as there will be dscussion or opinions will be 
sought), however these aspects are not explained and are not facilitated by either the assignment itself (what >mII be 
obsen/ed a why) or by the description of what will be done with the students' results. The assignment is mostly or 
totally dosed-ended and appears to have little purpose in the learning process. Any rationale for the assignment is either 
absent, nonsensical, or is under described and appears as an unsupported assertion. 

Level 2: Either the assignment or discussion may indude elements of pertinent observational learning (maybe even a 
novel approach), but usually not both. The assignment is not well developed, either being too understructured and vague 
or too overly structured and rigid. The assignment doesn't opennjp many learning possibilities and usually appears ovedy 
simplistic. Often, the analysis suggested for dassroom activities doesn't match the observation conceptually. 

Level 3: The assignment seems to reflect one of the following structural or organizational characterrstics: a) observa- 
tiorts involve quite a bit of student involvement in the topic, concepts, or observational procedures, but no explanation 
of what will be done with the data collected or the activities are developed while the observational data ( ithering Is not., 
b) both the student observation and data analysis or dassroom activities show some de^^ee of sophistication but each 
are either somewhat vague or somewhat overstmctured, c) observations do not seem to match the 
data analysis/activities weH but overall the assi^ment seems to have some depth or conceptual breadth or each is 
developed to some extent.The assignment seems to hold many possibilities and ideas are explained somewhat but not 
fully developed or elaborated upon. A general conceptual Idea, focus, or theme Is usually apparent although independent 
concepts within the assignment are not fully integrated together or related to one another. 

Level 4: The assignment is conceptually related between parts, although one or the other may be more devdoped 
conceptually. The Intent or purpose or focus of the assignment is dear and is explained or can be inferred. Some ideas 
are not elaborated upon enough to know or Infer that the students will reach a full understanding of the concepts or of 
the quality of thinking which ts relayed as an intended outcome. 

Level 5: The assignment ts conceptually broad and integrated The concepts are elaborated upon and many suggestions 
are given on how to Integrate the topic into the dassroom and how assignments might be related conceptually. A 
meaningful leaming experience and student involvement Is facilitated by requiring or allowing them to structure a major 
portion of their own leaming experience. 
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P3 - Individualizing Instruction 

Levd 1 : tndK/iduaKzing instruction typically entails dchotomlzing students into two abirty groups such as above 
average/bdow average, fast/slow or just focusing on characteristic of students. Assertions are made about what type 
of instruction, actj\^tjes, etc will be instituted but these are not described well and no rationale explaining the choices 
made is given. The recommendations focus on obvious, stereotypical, and/or inappropriate conceptions of learning. 
Often the recommendations focus on more or le» of something (Le. practice, pace, facts, essays, quizzes, 
etc.) 

Level 2: Indviduaiizing instruction may entail focusing on one or a few characteristics of students.OesGnptjons are 
gven about what type of instruction, activities, eta will be instituted and some simplistic explanations for the choices 
made may be given. Any overall purpose or focus is usually absent. The recommendations focus on simplistic concep- 
tions of leaming. The recommendations may focus on more or less of something (i.e. practice, pace, facts, essays, 
qMizzes, eta) but aiso on ideas of student interests and needs to some extent. 

Level 3: Indviduaiizing instruction entails focusing on characteristics of a student or students that are based on ways 
of leaming or leaming styles, etc, but typically not on dchotomous ability characteristics. Detailed descriptions may be 
given about personalized or independent instruction, activities, projects, eta that fadiitate meaningful leaming. The 
overall purpose or focus is usually to improve leaming, fadiitate student interests, oi' further oonceptions. The rationale 
or explanation given for what is to be learned is typicatty not well developed or may appear to focus on surface aspects. 
Some explanation may be 9ven for how the indviduaiized instruction fits in to the classroom. 

Level 4: Indviduaiizing instruction entails focusing on many characteristtcs of a student or students, the students 
needs, and the dynamics of the dassroom. Explanations are given about possible instruction, activities, etc that will be 
changed to fadiitate learning. The instruction typically fadlitates student problem solving and so structuring of the task 
v^ile fadlitating broad dassroom goals and/or subject matter objectives. The conceptions of indvidual dfferences in 
leaming may or may not be detailed or elaborated upon, but the acth/itles described opervup marry avenues for student 
interest, abilities, and leaming opportunities and focus on student oortstructions of leaming experiences. 

Levd 5: Indviduaiizing Instruction entails focusing on instruction which fadlitates a studerrts general problem solving or 
reasoning capabilities ^ich may have a global effect how they leam. The student is seen as a change agent involved In 
active partldpatlon in his/her development. The dianges in the dynamics of the d^^oem may Indude both short and 
long term teaching goals. Explanations are given about the possibilities for change and the effect of the instruction. The 
overall purpose is usually to faditate maximum student potential through analyzing their own constnictions. 

P4 - Self-Management 

Level 1 : Problems are viewed from a reactive stance with the emphasis being on dassroom management procedures 
and/or behavior being controlled by the teacher. Sdf-management is typically dscussed in tenns of student behavia 
anc! is seen as good/bad, right/wrong, compliant/noncompliant, or indcating a problem ^h not getting homework 
finished on time, being asodal, unorganized, etc Pedagogical concepts are dsaete and unrelated (dsjdnt) or only one 
particular concept is raised and dscussed. Littie explanation or understandng of the concept of self-management is 
relayed 

Level 2: Problems are viewed with an emphasis on dassroom management procedures and/or behavia being oontrdled 
by the teacher. Ar:^ions may be made that there is some contrd by the student but are not facilitated by other 
statements made. Self-management is typically dscussed in terms of student behavia and is seen as good/bad, right/ 
y^ong, compliant/noncompliant, a indcating a problem with not getting homework finished on time, being asodal, 
unaganized, etc Pedagogical concepts are simplistic Goals are fomrted and aaion monrtaed by the teacher. 
Understandng of the concept of self-management is limited 

Levd 3: Problems are viewed with an emphasis on behavia bdng monitaed by the teacher. Asseruons may be made 
about positive change occuring but the rationale is not explained a supported. Sdf-management is typically dscussed 
in terms of student needs or indcating a problem with not getting homework finished on time, bdng unaganized, a 
not understandng , eta Pedagogical concepts are conceptually understood but often somewhat sirnplistially rdayed. 
Goals 'ire formed and activities are monitaed by the teacher but goals are not rigidly defined 

Levd 4: Self-management is viewed mainly in terms of creating avenues fa leaming. Spedfic goals are usually co- 
created by the teacher and student. Sdf-management is typically dscussed in terms of student needs, leaming new 
coping strategies, and/a leaming how to leam. Pedagogical concepts are conceptually understood, integrated, and 
rdated to dassroom experiences/activities. 

Levd 5: Self-management is seen in broad temns of aeating avenues fa leaming, sdf-awareness, conf^ia resolution, 
and creating cooperative goals. Sdf-management typically implies emphasis on student needs, leaming to reason out 
problems, and maximum student freedom, partidpation, and responsibility fa the age group. The ideas rdayed typically 
involve avenues open to evolving change and therefore may not appear detailed. Pedagogical concepts are conceptually 
understood, integrated, and rdated to dassroom experiences/activities. 
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P5 - Guidng Students in Strategies for Learning 

Lcvd 1 : Cues appear behavioral and not coj^ltive in nature and focus on maxims, rules, or hints for cxxrect b«haw^ 
Correct behavior ts interpreted in terms of soda! appropriateness, classroom procedure/manaoemem or croo^^o 
complete assignments. The cues focus on superficial or surface information to be relayed by the teaciief i^dfrSrL-H 
by the student. ^ ana iwjwea 

Levd 2: Cues appear behavioral and not cognitive in nature and focus on maxims, rules, or Nnts to aid vi completion of 
a task or carry out an assisyiment. Behavior may be interpreted in terms of social appropriateness, dassroom proce- 
djre/management, or procedjre to complete assignments and/or may focus on superfidal or surface information about 
what ts being learned or >what the student should or could do. 

Level 3: Cues appear c»j^itive or affective in nature and focus on hints or aids to learning but are often conaete or 
surface onented. Cues mzy rdate to strategies a students could use but are not explained and it is dffiait to rfer a 
connection to meaningful .earning. Typically, there is no stated or implidt purpose, rationale, or conceptu^ framework 
devdoped cspedally as • Jated to how students may be conceptualizing subject matter. 

Levd 4: Cues are cognitive and/or affective in nature and focus on student learning and undenrt^xfrig on a conceptual 
levd. Typicalty, the cues are integrated on more than one levd. Cues may rdate to strateges a students could i»c 
although they may not be daborated upon there is a connection to meaningful leamina Typically, there is a stated or 
implidt purpose, rationale, or conceptual framework for the cues and these are inte^^ted into dassroom activities, 

Levd 5: Cues are cognitive and/or affective in nature and focus on student problem soMng and rdvidual construction 
of meaning. Qjes are integrated dynamkally into the dassroom environment mabling students to use them as tools 
thdr own learning, and fadlitate maximum meaningful learning. Typicaly, there are several stated or »np6dt ' 
purposes, a rationale is explained, and a conceptual framework for integrating the cues into dassroom activities is 
devdoped Muitilevd planning and considerations are designed. 

P6 • Perception/Techniques for Cognitive & Learning Styles 

Lcvd 1 : The stimulus chosen doesnt appear open to individual interpretations or students perceptions. Hypothetical 
response* are sometimes absent or are «mited in meaning. If a purpose or focus is stated. It is superfidal or ts often 
rdated to rote learning. Typically at this levd it appears the concept of "stimuli cpen to interpretations" has eithv been 
mttunderstood or is simplistically interpreted and the rest of the assignment cannot be devdoped 

Levd 2: The stimulus chosen may be open to indvidual interpretations or students perceptions, but the as»^yvnent 
may limit the range of interpretations. Hypothetical responses are sometimes absent or ve limited in mear^na A 
purpose or focus is typically stated, but it Is not daborated upon and It Is dfflcult to infer if meaningful leamina w« take 
place. Blanket assertions are often made about learning whidi are unsupported 

Levd 4: The stimulus chosen is open to indvidual interpretations or students* perceptions, Hypothcticai responses 
rdate the use of the stimuli to meaningful leaming, A purpose or focus is typical^ stated or can be infen-ed and 
instruction Incorporating student responses is described Activities or considerations may describe possiblities for 
further instmction, how students are thinking, and/or indcate a change in students conceptions, 

Levd 5: The stimulus chosen is open to indvidual interpretations or students' perceptions. Hypothetic^ responses 
rdate the use of the stimuli to meaningful leaming and broader dassroom goab, A purpose or focus is stated or is 
implidt and instruction Incorporating student responses on many conceptual levds is described. How students are 
thinldng and changing students* conceptions through various techniqfues is a goal of the exerdse. PossibiMties for further 
instruction are explained. 

P7 - Evaluating OpervEnded R-oblems 

Levd 1 : The question/problem itsdf may or may not be dvergent and open to indvidual interpretatiorw, Assertksns 
may be made that it is open however evaluation is based on correct or right answers. The evaluation oiteria are 
stmpiistic and focus on superfidal conceptions of quality, sometimes focusing only on quantity and are utsiMDported A 
rationale for the evaluation procedures may be absent. 

Levd 2: The (^estion/proWem is open to dvergent responses. Evaluation is based on a number of concerns most of 
which are rdated to knowledge, conceptions, or reasoning. The evaluation criteria are justified with superfidal maxrns or 
assertions about why they were chosen or are important but they not explained on any corxseptual basis, 

Lcvd 3: The (^estion/problem is open to dvergent responses. Evaluation is based on a number of concerns most of 
which are rdated to knowledge, conceptions, or reasoning. The criteria foon a conceptual whde and thdr purpose is 
stated or can be inferred but a rationale is not given In detail. Some other conriections to teaming or instruction^ 
methods may be desaibed. 
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Lev«i 4: The question/problem is open to diver gent responses. Evaluation is based on a set of concerns with a major 
focus on cntical thinking, developing conceptual networks, or problem soiving. The purpose is stated or can be inferred, 
related to the subject matter or dass goals, and a rationale is explained and somewhat supported Some other connec- 
tions to teaming or instructional methods may be described. 

Level 5: The question/problem is open to dvergent responses. Evaluation is c^agnostic, based on a set of concerns 
which focus on the student coftstructing understandings. The major purpose to encourage the development of thought 
while other purposes are related to the subject matter or dass goals. The rationale explains and supports the use of the 
criteria. Some other connections to learning and instructional methods are further described. 

P8 - Worksheet on Evaluating Mass Media 

Level 1 : The assignment appears irrelevant or misplaced in regards to the task as the form of the questions or con* 
cems stated are dosed-ended. are conceptually dsjointed between questions, are conceptually void within questions, 
and are not connected to any form of problem solving or critical thinking overall. Any explanation of students* thinking, 
conceptualizations, or purpose is usually absent but may appear in the form of unsupported and simplistic assertions. 

Level Z: The assignment appears somewhat irrelevant, misplaced, or ill-conceived in regards to the task as some 
questions require no problem solving or critical thinking although some may be conceptual in nature but not require any 
analysis on the students' part. There is usually no uni^ng idea or conception aithou^^t sometimes the assignment will 
appear to be based on a particular topic, subject, or simplistic purpose. The purpose may appear vague, restricted, or ill 
conceived even when stated explldtty. No indication is given or can be infenred or th« conceptual or instructional 
importance of the assignment. 

Level 3: The assignment often has a unifying theme or focus which relates ail questions or concerns into a whole set of 
kistruction. Within questions or concerns some conceptual breadth and focus on problem solving is apparent. Overall, it 
is not apparent that the instruction would foster critical thinking and problem solving by encouraging student construe- 
tkxis or interpretations of meaning. Some explanations are given for the purposes as rdated to learning, but are often 
underdeveloped and may appear as descriptive but not explanatory. 

Level 4: The assignment has a unifying theme and purpose and is process oriented encouraging the student to engage 
ki active problem solving and/or critical thinking. The questions or concerns are conceptually weU-deveioped and ask for 
probable or possible causes and solutions constructed by the student. An explanation of the analyses and evaluation of 
student responses is given. 

Level 5: The assignment has a uni^ng theme and purpose and is process oriented encouraging the student to engage 
ki active problem solving and/or critical thinking within a rich context. The questions or concerns are probing in nature, 
fom a conceptual networic related to the context, and ask for probable or possible causes and solutions constructed by 
the student. The analysis and evaluation of student responses is explained and elaborated upon in relation to dassroom 
processes and other aspects of teaming and instmctton are incorporated (e.g., indvidual needs, connections to 
other assi^iments. connections to broad educational goals, etc) 

P9 - Evakiating Student's Thinking Processes 

Level 1 : Criteria may be looking for faaual and convergent information, and do not focus on indvidual needs or proc- 
esses. If there is any iocvs on processes it is limited, i.e. mnemonics, and not necessarily connected to how the student 
is thinking. Any explanation gpven seems to reiterate the focus on convergent answers or a focus on a definite end 
pant, a go no go situation, either the student performs the skiH or doesn't. 

Level 2: Criteria may appear conceptual in nature, concerned somewhat with how students are thinking, however any 
focus on processes is limited and the conceriis are not described or explained The concerns and any explanation given 
put limits on thinking as a constructive process turning the assignment as a whole into a dosed-ended experience. As a 
whde. the assignment may show preliminary concerns for viewing thinking from dfferent aspects. 

Level 3: Criteria are conceptual in nature, concerned with how students are thinking. The concerns raised and any 
explanation given describes student thinking as a constructive process, however these aspects are not pursued How 
the criteria would be integrated into instmction is not described in detail but some consideratior^ may be raised 
Levd 4: Criteria are conceptual in nature, concerned with how students are thinking. The criteria and explanatkxi given 
describes student thinking as a constmctive and problem solving process, some aspects are pursued How the criteria 
would be related to or are inte^-ated into instmction is described and some considisrations may be raised and dscussed. 

Level 5: Criteria are conceptual in nature, concerned with how students are thinking, general thought processes, and 
critical thinking or problem solving and aspects of incivldjal dfferences are integrated to dagnose student strengths 
and weaknesses. The purpose of the criteria is to formulate a picture of each students ways of leaming and reacting to 
situations. How the aiteria would be effect instruction is described and various considerations are raised and dscussed. 
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P10 • Ran to Prevent Dtsopline ProWems and Create a Positive Environment 

Level 1 : ProWems are viewed from a reactive stance with an emphasis on dassroom management procedures and 
contrdling behavwr. Stuc int behavior is seen as good/bad, right/wrong, compliant/noncompliant Creating a positive 
environment entails attention to conaete/physical aspects of the dassroom. Pedagogical concepts are dscrete and 
unrelated (dsjoint). No explanation cr understanding of intended purpose of methods is apparent. 

Level 2: Problems are viewed from a reactive stance with the emphasis being on dassroom management procedures 
and controlling behavior. Student behavior is mainly seen as good/bad. right/wrong, compliant/noncompliant. Creating a 
positive environment may entail attention to conaete/ physical aspects of the dassroom and aspects related to 
leaming (sodal). The learning environment may be autocratic in nature although dassroom management and mle 
setting are not Some explanation and/or understanding of intended purpose of methods is apparent however, the 
pedagogical ideas are not integrated 

Level 3: Problems are viewed from a mainly proactive stance with the emphasis being on dassroom management 
proceckjres vp^ich help prevent problems and less emphasis on rule based behavior. Creating a positive environment may 
entail attention to conaete/physical aspects of the dassroom, aspects related to leaming (sodal and/or cognitive), and 
dassroom management. Some explanation and/or understanding of intended purpose of methods is apparent however, 
the pedagogical ideas may appear integrated 

Level 4: Problems are viewed from a proactive stance with the emphasis being on dassroom management procedures 
(\^ch may faditate dassroom interaction). If disdpline is mentioned it is generally not punitive and is instead based on 
natural consequences. Creating a positive environment may entail attention to conCTetc/pJ^cal/sodal aspects of the 
dassroom, aspects related to leaming (physical, emotional, sodal, cognitive), and dassroom management Many 
(Afferent Ideas or concerns are described Some explanation and/or understandng of intended purpose of methods Is 
apparent and pedagogical ideas are usually integrated or not conflicting. 

Level 5: Problems are viewed from a proactive stance with the emphases being on dassroom management procedures 
while fadiitating interaalon, individuality, and awareness of dassroom dynamics. Rules or d sdpltnt nr^ay be mentioned 
but in a non-punitive manner and are typically based on student involvement. Communication and Interest in leaming are 
most often seen as the best preventative methods. Major emphases for aeating a positive environment are setting up a 
warm, caring, and open atmosphere and concern for students needs although other aspects may beconsidered as well. 
Methods and pedagogical ideas are integrated into an holistic pi^ure of the dassroom. 

P1 1 - Ran to Incividuaize Instmction 

Level 1 : Individualizing Instruction typically entails dchotomizing students into two ability groups such as above 
average/below average, fast/siow or just focusing on one charaaeristic of students. Assertions are made about what 
type of instnjction. activities, etc, wil be instituted but these are not described well and no rationale explaining the 
choices made ts ^en. The recommerxiations focus on obvious, stereotypical, and/or inappropriate conceptions of 
learning. Often the recommendations focus on more or less of something (i.e. practice, pace, facts, quizzes, etc) 

Level 2: Indvidualizing instruction may entail focusing on one or a few characteristics of students. Descriptions are 
given about what type of instmction, activities, etc. will be Instituted and some simplistic explanations for the choices 
made may be gpven. Any overall purpose or focus is usually absent. The recommendations focus on simplistic but 
appropriate conceptiofts of learning. The recommendations may focus on nr>ore or less of something (i.e. practice, pace, 
facts, essays, quizzes, etc) but also on ideas of student interests and needs to some extent. 

Level 3: Indvidualzing instruction entails focusing on several characteristics of a studerrt or students. Descriptions are 
0ven about instruction, activities, etc will be changed to fadlitate individual dfferences. The overall purpose or focus is 
usually to improve leaming and/or motivation. The recommendations focus on many conceptions of individual differ- 
ences in teaming. Although these may not be detailed or elaborated upon, some detail nuy be given about changes in 
actnnties. 

Level 4: Indvidualizing instruction entails focusing on many characteristics of a student orstudents, the resultant needs 
fa change, and the dynamics and complexity of the dassroom. Descriptions are gven about possible instruction, 
aarvities, etc, that will be d^anged to fadlitate leaming. The overall purpose or focus is usually to fadlitate dassroom 
goals and subject matter objectives while tailoring plans to individual differences, realizing that some students may not 
succeed at the same tasks. The conceptions of individual differences in leaming may or may not be detailed or elabo- 
rated upon, detail is gven about possible changes in activities. 

Level S: Indvidualizing Instruction focuses on many characteristics of a student or students, the resultant need^ for 
change, and the dynamics and complexity of the dassroom. The student is seen as a change agent involved in making 
decisions about his/her leaming. The changes in the dynamics of the dassroom may inducte both spedfic and global 
examples. Explanations and details are given about possible instruction, activities, etc that will be changed to fadlitate 
leaming. The overal purpose or focus is usually to fadlitate maximum student potential while obtaining dassroom goals 
and subject matter objectives. The respondent indcates that some students may not succeed at the same tasks, but 
also that each student may have unique capabilities. 
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Appendix D: Examples of Rationales for Levels of Pedagogpcal Uhderstancing 



PI : Convergent and Divergent Questions 

Develop meaningful convergent and dn/ergent questions covering one topic and state what the purpose of each 
question is in learning. 

Level 1 : The c^estlons, when seen together, do not form a conceptual whole and may not be related except possibly 
by subject area. The majority of the convergent and dvergent questions lack conceptual breadth and are stmpltstic, i.e., 
the divergent question may didt only a few words, possibly a one word answer wtiich may even be known, or may only 
be open to limited interpretations. 



The convergent questions are to test the students' ability to remember important facts. The dvergent ques- 
t\ors test the students* ability to think ^y these events are important. 

In this response, there is no conceptual connection made between the dfferent questions. The convergent 
questions only ask for a one-word, correct answer tc test the memorization of facts. The dvergent questions simply 
turn the convergent questions into more open*ended forms, although it is dffiajtt to tell if the respondent would 
consider dfferent interpretations. Furthermore, the dvergent questions appear misguided, confusing, and are not 
stated in such a way that would elidt the intended outcome, **... students' ability to think about why these events are 
important.** With no explanation of how these questions would be used in the dassroom, it is dfficult to know if they 
would have any instructional value. 

Level 2: Most of the questions are lacking conceptual breadth and are often phrased stmplisticalty, state an obvious 
purpose or are lacking a purpose. The questions typically fit conceptually together, only in the sense that they all cover 
the same topic, but are only vaguely related otherwise. Some explanation may be given for the purpose behind the 
questions. 

Example: Topic - Early American History 
Convergent Questions 

1 . Who was the commander in c^ef In the South during the Civil War? 

2. Who was the author of Tetters From a Pennsylvania Farmer?*' 

3 Laws regulated the colonies commerce during the 1 700's and imposed high duties and tariffs on non- 



Divergent Questions 

1 . What do you think would have happened if the British would have won the War for Independence? How do you 
think our lives would be dfferent? Please give examples. 

2. If you could be any known person during the War for Independence, who would you be and why? 

3. Describe how the mercantile system affected the American colonist How do you think this sfystem affected our 
future development of a free market? 

. These questions are aH loosdy related to the same topic, early American history, but do not form a conceptual 
whde (although some of the questions seem conceptually rdated). There is a slight variatkxi on the way convergent 
questiorts are asked (fill in the blank) and the type of processing which may be required to answer the questiorts. The 
f vergent questions are a bit more thought provoking and conceptually orientated (with fdk^w-up questions to eiidt 
more than a one word response) than most typical level 2 responses, however there is no explanation of how these will 
be used in instruction or what the purpose are as rdated to learning and the subject content. 

Levd 3: Questions are all rdated conceptually to the same theme or topic. Attempts are made at identifying and 
explaining the purpose behind the questions. Most questions are conceptually missing interim links even if related to a 
partiaHar topic and theme, however some of the questions do appear to be linked conceptually. 

Example: Cdl Reproduction 



1 . In all organisms, the sperm and egg cdls contain half the number of chromosomes found in the body cdl (T /F) 

2. Mdosis is the process through which cdls split into two dipidd cells (T/F) 

3. Dipidd daughter cdis contain the same number of chromosomes as the parent cdl from wNch they were formed. 



Example: 

A. Convergent 

1 . When was the Dedaration of Independence signed? 

2. Who was the first president of the United States? 

3. Who won the Battle of Bunker Hiy? 



B. Divergem 

1 . Why was the Dedaratk)n of Independence si^ed? 

2. Why was Washington the first president? 

3. Why dd the U.a win the Battle of Bunker Hill? 



English goods. 



Convergent: 



a/F) 
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These are achievement questions to determine what the student is comprehendn^ in this area of cells, mainly 
cell reproduaion, and to determine if an area needs to be explained better or in a different way. 



Divergent: 

1 . Why does a reproductive cell (spcmi or egg) need to have one half of the chromosomes of a body cell? Back up 
your reasons with facts along with your own theory. 

2. Why is there segregation of alleles and what does segregation of alleles cause to occur? 

3. Why do you think there are only four nucleotides? 

These questions are also achievement questions, but they are not wanting the student to give memorized facts. 
They are designed to make the student think about the concepts in a broader light instead of allowing them to 
accept the facts as they are. These questions promote creative thinking and the Idea that facts can be questioned. 

These questions are ail related to the conceptual understanding of one complex topic, cell reproduction. While the 
convergent questions are ail true and false, which means students could guess at the answers, they are conceptually 
stated so that a student could think through the answer. The divergent questions focus on important concepts, 
processes, and cause and effects related to cell reproduction and are thought provoking. Attempts are made at 
identifying the general purposes for each question type but is not explained in terms of a conceptual understanding of 
the subjea content or the instructional process. 

Level 4: There is an overall topic or theme or purpose. Convergent questions are formed on a more conceptual basis 
instead of rote answers. As a whole, the assignment seems meaningful and conceptually stnjctured. Questions lead to 
more understanding of the topic as a whde and purposes may also seem more broad than at the previous level but are 
still vaguely defined in some cases. 

Example: Discussion questions - The Egyptian Qnderella" 

Divergent Questions: 

1 . How do you think this book Is different from the Cinderella story that you are familiar with? purpose * predicting 
information 

2. Do the pictures in this book help you to imagine what it is like to live in Egypt? Do they seem real? Why? (dver- 
gent answers here will be based on the last question pnmariiy) purpose - attention to visual details, support of 
opinions 

3. Which of the two Onderella stories did you like best and why? purpose - support opinion by backing it up with 
facts 

Convergent Questions: 

1 . How did the falcon help Rhodopsis? (he dropped her rose-red slipper in the Pharaoh's lap) purpose - detail 
question 

2. Rhodopsis did not have mean stepsisters in this story. Who was not very nice to her that replaces the stepsis- 
ters? (the Egyptian servant girls) purpose - comparison with familiar Gnderdla story, detail question 

3. Why do you think that water was so important in this Cinderella story? (because Rhodopsts lived by the Nile 
river) purpose - inference question, comparison with familiar story. 

The overall purpose of these questions is to test reading compreher^sion. 

These questions all relate to an overall theme, comparing two similar stories, chosen as a vehide for the purpose, 
evaluating reading comprehension. Each question has been identified as assessing important aspects of reading compre- 
hef«ion although these are only vaguely identified. The convergent questlor« are formulated to help darify. detail, and 
make conceptual comparisons instead of testing memorization. Divergent questions focus on various aspects of 
reasoning and processing information. Although the questions could be appropriate in either individual or group instruc- 
tion no details are given about how this process would take place or how reading comprehension could be fadlftated 
through instruction. 

Level 5: There is an overall theme or purpose. Convergent questions are formed on a more conceptual basis as in the 
previous level and they are conceptually interrelated to divergent questions, one question (of either type) is used as a 
lead-in to the next question, or questions bring up different elements (subthemcs) important to the theme (there are 
possibly other organizational/structural diararterlstics which may be manifested in other assignments at this level). 
Often, explanations are given for why one question is used in a particular point in the instruction and what other aspects 
of instruction or method would be implemented upon use. 

Example: Convergent and Divergent Questions 

Let's say that I'm going to introduce James Joyce's short story "The Sisters" to a senior level literature dass. 
The day before, I introduce the topic by giving some background knowledge about the format of traditional short 



stories and then moved on to introduce James Joyce as a prolific representative of this genre. I would then ask 
them to read The Sstcrs" for the next dass period, keeping in mind the format of traditional short stories as they 
read the text. 

At the beginning of the next dscussion period, I rould start the discussion off with basic convergent questions 
to test surface level comprehension. 

Q: Who is the protagonist of the story? Who is the antagonist? 

A: the boy; the deceased priest 
These opening questions serve to get the ball rolling and test literary vocabulary. After a few more of these type. I 
could then move on to more complex convergent questions. 

Q: What did the boy dream about? What was the priest's transgression? 

A: he dreamed about the priest confessing to him after his death; he wasn't to be found when needed 
These convergent questions are important for two reasons: they help to understand the surface ideas of the story 
and then lay the groundwork for understanding the deeper symbolic level of the story. This symbolic level can be 
explored by using cSvergent questions 

Q: Why do you suppose the protagonist's name was never mentioned? 

A: any thoughtful respof«e is correct. 
By comparing the surface level to the students* thoughts about the divergent questions, interesting points of view 
toward the story's symbolic meaning will be reached. 

Q: Why does the boy fed joy instead of sorrow at the priest's death? 

A: any thoughtful answer is correct 

Q: What do you think the boy's dream means? 

A: any thoughtful answer is correct 
Of course, many more of both types of questions are needed to get a good comprehension of the story as a 
whole. But by asking these types of questions, they will hopefully think about the topic and maybe even begin to 
question for themselves when they read. It is a guided discovery using scaffolding into the intricades of The 
Ssters** that might make the students more aware of their own reality. That's why I love literature. 

In this response, th^ overall theme is understanding and evakiating tradtional short stories, using one story as an 
example to explore related concepts. The questions are integrated into a set of instruction where a logical process of 
working through meanings is implicated and is based on ideas about how students leam and construa ideas. Both 
convergent and divergent questions are conceptually interrelated to the theme and one question (of either type) is 
used as a lead-in to the next quef-vion. a questions bring up different elements important to the theme. Explanations 
are given for why the questions ^re used at particular points in the instruction and other aspects of leaming are 
discussed such as scaffdcing: symbolic meaning, and students leaming to question for themselves. 

P6: Perception/Techniques for Cognitive & Leaming Styles 

For this assignment, pick out a stimulus whidi will be presented to your students which is open to interpretations. 
Describe the different responses you would get to this stimukis. Explain what the next step(s) in your instruction will be. 

Level 1 : The stimulus diosen doesn't appear open to indvidual interpretations or students perceptions. Hypothetical 
responses are sometimes absent or are limited in meaning. If a purpose or focus is stated, it is superfidal or is often 
related to rote learning. Typically at this level it appears the concept of "stimuli open to interpretations" has either been 
misunderstood or is simplistically interpreted and the rest of the assignment cannot be developed. 

Example: The stimulus I have diosen to present to my students will hopefully encourage them to be concerned 
about their dassroom assignments. I will be issuing vocabulary tests eadi week in my dass. My purpose is to help 
my students to leam important words used in our language and culture. My purpose is not to have more assign- 
ments to make up their final grades. If a student misses more than zero on any vocabulary test, they must retake 
that test. They will continue to retake the exam until they miss zero. I want to help my students expand their 
vocabularies. I will begn these tests at the start of the school year and continue them until the half-way pdnt of 
the term. Starting the second half I will begin these same tests again. The purpose of this is to make sure the 
students leamed the vocabulary words well enough to use them in their daily lives rather than leaming them only 
well enough to pass the test. The individual stimulus would be the retaking of the tests. 

Various responses which might occur 

1 . The ideal response would be for all students to work hard at improving their vocabulary skills in order to score 
well on the tests. Hopefully, each student would pass the test the first time in order to move on rather than having 
to continually retake tests. It would also be ideal for the students to remember the words in order to pass the tests 
the second time around. This will tdl if they really knew them or whether they only studied wdl enough to pass the 
test the first time using their short term memories. 

2. Another response could be that the student could work at passing eadi test but soon after forgetting the infa- 
mation causing them to not pass the second round of tests. 

3. Still another response could be that the student does not pass any of the vocabulary tests and continually falls 
behind because he/she has to keep retaking eadi test. In this case, the student would prob;ibly need one-on-one 
help with the subjert matter. 



The standard of passing is that of missing zero. This project would be used at the high school levd. 
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The stimuli for this response, the vocabulary tests or "the retaking of tests**, presents a passive situation and 
a doscd-ended stimuli. The students* strategies in memorizing vocabulary words and how well they perform in is the 
focus instead of focusing on hypothetical student constructions. A focus on the conceptual or implicit meanings of 
words and less focus on shear mastery would have enabled the respondent to formulate hypothetical interpretations 
that students may construct. This change in focus could have lead into further activities to develop understandngs as 
related use in daily life instead of leading into a pass/no pass dchotomy in evaluating learning. 

Level 2: The stimulus chosen may be open to individual interpretations or students' perceptions, but the assignment 
may limit the range of interpretations. Hypothetical responses are sometimes absent or are limited in meaning. A 
purpose or focus is typically stated, but it is not elaborated upon and it is difficult to infer if meaningful learning will take 
place. Blanket assertions are often made about learning which are unsupported. 

Example: For a aeative writing assignment I would use a page that had incomplete sentences which the student 
would have to fill in with their own made up words. This cxerdse would help students learn nouns and verbs plus 
encourage their writing skills. 

Today I went to the and saw a doing various moves In a There were many aowded around 

the They were all and Then, a appeared and to everyone. This was not what I wanted, so 

I and went to . 

In this response, the stimuli is open to indvidual interpretatiorw. but tehy are limited by the sentence structire. 
ProN^dng some structure for students In this form may result in learning nouns and verbs (probably learning the correct 
placement of nouns and verbs). It is unknown If the supplied structure would actually aid learning as it may be an 
impediment, being too complex and confusing for students who do not already know sentence structure well. Me^ingful 
learning may be encouraged more by allowing students to make up their own sentences. No hypothetical student 
responses are supplied nor is there a discussion of how this activity would fit in with instruction. 

Level 3: The stimulus chosen is open to indvidual Interpretations or students perceptions, hlypothetlcal responses 
confirm that the stimuli is meaningful and open to different interpretatiofw or perceptions, A purpose or focus is 
typically stated or can be inferred but may not be elaborated upon and it Is dfficult to infer all of the steps In the 
learning process. Typically activities or consideratiorts descnbe possibilities for further Instruction. 

Example: Stimulus: slide show Illustrating the various recently proposed techniques fa waste dsposaL ex: sending it 
invo space, burytng it, dumping it into the ocean, etc. 

Hypothesized Responses: I would observe the students* fadal expressions to see what kinds of responses the slides 
produced, ex: did they look shocked, repulsed, amused, or unconcerned, etc, Snce this would be for an Engksh 
dass, I would have each pick one of the discussed methods of disposal, research It. and write a paper on it (based 
on their reaction to the slide-show, they can give their opinion on the method if they desire). 

The stimulus Is open to many different interpretations and using the stimulus in instruction opens up avenues 
for other meaningful activities. The hypothetical responses presented represent a somewhat limited view of how 
students could react. The purpose of presenting this stimulus could have been broadened by presenting one or 
more thought provoking questions in conjunction with the slide show to observe students* initial verbal reactions in 
addition to their fadal expressions. Otherwise, it is possible the students would have little reaction because of prior 
exposure but may have opinions and concerns which they have never voiced. Student responses are not fully 
incorporated into the Instruction. 

Level 4: The stimulus chosen Is open to indvidual Interpretations or students* perceptions. Hypothetical responses 
relate the use of the stimuli to meaningful leaming. A purpose or focus is typically stated a can be inferred and 
instruction incorporating student resportses is described. Activities or considerations may describe possibilities for 
further instruction, how students are thinking, and/or indicate a some change in students conceptior^. 

Example: [the stimulus shown is a boy with hands in his pockets, walking along. He looks sad. Above him is an 
empty caption such as in a cartoon which represents what he Is thinking.] Tell me what you think this young boy is 
thinking. Pill In the caption with your response and tell why you think he*s thinking about It. 

Grades - 6th and up 

The students are gang to answer the question and tell why they answered that way. Some of the possible re- 
sponses might be: 

- his dog died 

- he got a bad grade on a test 

- he had a fight with his friends 

- he forgot his lunch money 

Next. I will show the students my second picture which was to have taken place before the first picture. 
[Rcture shows same bey with a concerned look on his face. Behind him is another boy who is angnly saying 
something to him and who has denched fists. There is an empty caption over the second boy's head. It looks like 
the second boy may be somewhat older/bigger]. 1 will ssk the students now to fill in both captions. The responses 
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should change to maybe: 

- a fight with Ns big brother 

- an older boy being mean to him 
-etc 

Then we wiil discuss as a class why the younger boy might be upset and what he is thinking and compare 
responses. 

This stimuH are open to many interpretations by students and, as indcated by the hypothetical responses, serve as 
projective devices enabling the teacher to to find out more about the students in a manner in which the students would 
fed safe to respond The activities lead to meaningful learning as the students share their conceptions about how they 
interpreted the situation. In this case, the learning integrates both affective and cognitive responses to real life situ- 
ations and prcblem-soiving. The different presentation of the two stimuli would also allow the students to see how 
perceptions change based on knowing more evidence. 

Level 5: The stimulus Ghosr^n is open to indvidual interpretations or students* perceptions. Hypothetical responses 
relate the use of the stimuli to meaningful learning and broader dassroom goals. A purpose or focus is stated or is 
implicit and instruction incorporating student responses on many conceptual levels « described. How students are 
thinidng and changing students* conceptions through various techniques is a goal of the exercise. Possibilities for further 
instruction are explained. 

Example: For this assigiment I would show the children two stimuli, related to each other. The first one will be a 
vefy descriptive, short essay, i ^11 read to them about the beautiful and cultural aspects of the dty of Detroit, 
without ever mentioning the word Detroit, hiext, I will show them the second stimuli. This will be a set of pictures 
showing the bad, slum areas of Detroit. After showing the photographs, I will ask the students to write a short 
essay on what they just saw and heard. 

There could be many dfferent responses to these stimuli: 

A. The children will believe that this is only one dty and write about how there is good and bad points to every dty 
(and situation). They ^11 probably put more emphasis on either the good or the bad aspects dependng on what 
stimulus persuaded them more and their background attitudes already formed about big dties. 

B. They will believe that XhQ stimuli represent two different dties. They will talk about the evils of one and the good 
of the other. 

C. They will be persuaded mainly by the good aspect stimulus and write about the positive aspects of a dty, paying 
little or no attention to the negative aspects. 

D. They w^ii be influenced mainly by the pictures shoving bad aspects and write about the negative aspe<;;ts of the 
dty, paying little or no attention to the positive aspects. 

Reviewing these responses, I could use them to teach the children a lesson on how everyone has a dfferent 
response to the same situation and ask them why that is. I could also plan a lesson on how such drastically 
dfferent situations can exist in the same dty. 

Overall, this would be a good lesson for a ninth grade essay writing dass. This stimuli could be used as a part of 
a dty unit, descriptive unit, a photography unit (a picture is worth a thousand words) or a unit showing peoples' 
dffering attitudes. 

In this response, the stimuli are not only open to intefpretations« but as in the Level 4 response are set up to 
invoke thinking. The respondent dscusses how the responses could be used in many dfferent ways to expand students' 
conceptions about how people learn and are dfferent, about a variety of subject matter concepts, and about general 
strategies in learning related to dfferent modes of expression. The instruction also open to accomplishing broader 
educational goals sudi as learning about broader sodal and economic influences on people as a way of vie^ng and 
interpreting situations. 




Appendix E: Tables 



TaWe 1 : Means and Standard Deviations for All Participants' Ratings on Levels of Pedagogical 
Understandng. 



Portfolio Assignment 


Mean 


SD 


PI Convergent/Divergent Questions 


2.79 


1.13 


P2 Role of Observational Learning 


2.88 


1.25 


P3 IncKviciualizing Instruction 


3.34 


1.24 


P4 Self-Management 


2.89 


1.31 


P5 Strategies for Learning 


2.67 


1.24 


P6 Perception Techniques 


3.56 


1.31 


P7 Evaluating Open-Ended Problems 


2.97 


1.07 


P8 Evaluating Mass Media 


3.38 


1.24 


P9 Evaluating Thinking Processes 


3.34 


1.34 


PI 0 Creating a Positive Environment 


3.42 


1.37 


PI 1 Plan for Individual Instruction 


3.16 


1.31 



Table 2: Means, Standard Deviations, and Frequency Distributions for Ratings on Levels of 
Pedagogical Understanding for the First Half and Second Half of Semester Partitionings. 







First Half 




Second Half 






X = 


2.90 




X = 3.36 






SD 


= .81 




SD-.90 


Means 


n 




Percent 


n 


Percent 


1 <X<2 


8 




12.12 


4 


5.71 


2<X<3 


26 




39.39 


14 


20.00 


3<X<4 


21 




31.82 


29 


41.43 


4<X<5 


11 




16.67 


23 


32.86 



50 
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Table 3: Participants' Responses on Self/Course-Evaluations in Percent Responding and Average 
Number of Responses per Respondent 



Positive Negative 
Percent Average Percent /Average 



Teaching as a profession 


62 


2.9 


0 


0.0 


Kespeci/ Knuvmagc 


19 


20 


0 


0.0 


Commitment to teaching 


11 


1.3 


0 


o!o 


Teaching capabilities 


30 


1.1 


0 


0.0 


Conceptual understandkig 


46 


1.9 


0 


0.0 


Cognitive Development 


89 


1.9 


3 


1.0 


ueveiopmeni oi reasaiHrig 




1 8 


0 


0.0 


Structuring situations 


32 


1.0 


0 


0.0 


Relevance to future teaching 


43 


1.6 


0 


0.0 


General relevance 


16 


1.0 


3 


1.0 


Personal Growth 


59 


1.8 


0 


0.0 


Self-esteem and confidence 


27 


1.2 


0 


0.0 


Self-direction 


24 


1.1 


0 


0.0 


Self-awareness 


43 


1.2 


0 


0.0 


Attribution of change 


87 


2.5 


3 


1.5 


Open-Ended Assignments 


41 


1.4 


3 


1.0 


Course in general 


38 


1.0 


0 


0.0 


Time, effort, and difficulty 


65 


1.6 


8 


1.0 


Peers/mentors 


22 


1.4 


3 


2.0 


Course in General 


70 


2.0 


4 


2.4 


Course structure 


27 


1.3 


8 


1.7 


Compare to other experiences 


39 


1.5 


0 


0.0 


Feelings about the course 


46 


1.2 


5 


1.5 



Table 4: Correlations of Scores on the Inventory of Learning Processes vwth Ratings for Levels of 
Pedagogical Understandng, 



inventory of Learning Processes 

Fact Study Analysis/ Elaboration 

Retention Methods Synthesis 



Portfolios 

Total (n = 68) .13 .50*** .42*** .27* 

First Half (n = 64) .19 .49*** .42*** .32** 

Second Half (n = 67) .05 .41*** .36** .15 



* p < .05, ** p < .01 , *** p < .001 



